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METACOGNITIVE MISCALIBRATION 
AND UNDERACHIEVEMENT IN A 
COMPUTER LITERACY COURSE: 
SOME PRELIMINARY OBSERVATIONS 



Deborah K. Smith 
East Carolina University 



William Wittman 
East Carolina University 



C. Bryan Foltz 
East Carolina University 



ABSTRACT 

Metacognition, literally thinking about thinking, is a term used by cognitive psychologists to refer to our ability to monitor our 
own performance on cognitive tasks. The term also addresses the ability to assess our level of knowledge and skill in a given 
domain. Behavioral and psychological researchers frequently solicit metacognitive judgments from research participants in the 
form of self-assessm ent survey items. However, evid ence sug gests that ourmetacognitive judgments are often at odds with reality. 
The term metacognitive miscalib ration is used to refer to this disparity between self-assessments and more objective measures 
of ability and performance. The authors have theorized a relationship between metacognitive miscalibration and 
underachievement in courseswhere many students enter the class believing they already know the material. This paper reports 
preliminary results from an ongoing study seeking to understand the relationship between metacognitive miscalibration and 
undera chivem ent in a computer literacy course. 



INTRODUCTION 

“Professor Smith, I want to talk to you about my exam. I’m 
really surprised at my grade. I mean, I come to class everyday. 
I really know this material. I took computers in high school. 
And I feltreally good about the exam. I just don’t understand 
how I could have gotten such a lowgrade. Is there any chance 
there was some kind of mixup in the grading or something? I 
Just know I didn’t fail that test.’’ 

What causes stud ents to m aintain sue h a high opinion of their 
performance in the face of overwhelming evidence to the 
contrary? Is it possible that this phenomenon in some way 



impairs the studen t’s ability to learn? If so , might it be possible 
to correct the student’s misconceptions and, thereby, improve 
learning? 

These questionsform the basis ofan ongoin g research project, 
originally proposed in Smith and Foltz (2000), examining the 
relationship between metacognitive miscalibration and 
underachivem ent. The goals ofthis research are to develop an 
instrument that will identify students who are highly 
miscalibrated, examine the relationship between miscalibration 
and course performance, search for ways to recalibrate 
students’ metacognitions, and, finally, determine whether 
recalibration improves course performance. 

I 
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METACOGNITIVEMISCALIBRATION 



RESEARCH METHOD 



Metacognition is not to be confused with affect. Some early 
reviews of this work suggested including studies of students’ 
attitudes toward computers in the literature review. While 
affect may influence metacognition, whether students like 
computers or feel good about using computers, the subjectof 
many MIS educational studiesnot cited here, is not the subject 
of this research. 

Metacognition, literally thinking about thinking, is aterm used 
by cognitive psychologists to refer to our ab ility to monitor our 
own performance on cognitive tasks. The term also addresses 
the ability to assess our levelof knowledge and skill in a given 
domain. Behavioral and psychological researchers frequently 
solicit metacognitive judgments from research participants in 
the form of self-assessment survey items. However, evidence 
suggests that our metacognitive judgments are often at odds 
with reality (Kruger & Dunning, 19 99). The term 
metacognitive miscalibration is used to refer to this disparity 
between self-assessments and more objective measures of 
ability and performance. 

Psychological researchers have several theories to explain 
metacognitive miscalibraton. At the cognitive level, the cue 
familiarity theory (Metcalfe et al., 1993) essentially restates 
theold cliche, “a little knowledge is adangerous thing.” When 
people have some knowled ge of the domain in question, they 
are likely to have higher metacognitive judgments than when 
they have no knowledge of the domain. At the social level, the 
above averageeffect(Dunning et al, 1989; Alicke et al., 1995) 
simply says that people have higher opinions of themselves 
than of others. These self-serving assessments tend to increase 
with the amb iguity of the trait being assessed (Dunning et al., 
1989) and the levelof abstraction in the comparison (Alicke et 
al., 1995). 

Regardless of the source of metacognitive miscalibration, 
psychological and educational researchers have related degree 
of miscalibration to both prediction of performance on 
multiple-choice exams (Sinkavich, 1995) and actual exam 
performance (Shaughnessy, 1979). There seems to be general 
agreement that competence begets more accurate 
metacognitions (Maki et al, 1994; Kruger & Dunning, 1999; 
Shaughnessy, 1 979; Sinkavich, 1 995). One study suggeststhat 
gaining competence in the domain is the only way to correct 
metacognitive miscalibration (Kruger & Dunning, 1999). 

Of more practical concern is research suggesting that 
metacognitive judgments influence decisions to continue work 
on a problem (Metcalfe, 1998) and studying (Bjork, 1996). 
This being the case, improving metacognitive judgments - 
metacognitive recalibration - should produce better resu Its in 
courses where students are highly miscalibrated. However, if 
Kruger and Dunning are correct in their assertion that only 
domain competence will result in recalibration, then we are at 
an impasse. 



A 50-item multiple-choice quiz was developed as a 
comprehensive test of course knowledge. Questions covered 
eleven subject areas, with no fewer than four and no more than 
six items in each area. Ofthe five possible responses for each 
item, the last was “I don’t know,” and the remaining four did 
not include any obviously incorrect choices. (For example, 
when asked who is credited with designing the first computer 
mouse, Walt Disney would be considered an obviously 
incorrect choice.) Four metacognitive instruments were 
developed, soliciting students’ self-assessments oftheir course 
knowledge relative to their peers. 

Participants were students enrolled in two summer sections of 
a computer literacy course, taught by the same faculty 
member. A total of33 students completed thecourse pretest; 
23 completed theposttest. The course pretest, administered on 
the first day of class, consisted of the first metacognitive 
instrument, followed by the multiple-choicequiz, followed by 
the second metacognitive instrument. The course posttest, 
administered on the last day of class, prior to the final exam, 
consisted of the third metacognitive instmment, followed by 
the multiple-choicequiz, followed by the fourth metacognitive 
instrument. Students were instructed to choose the “don’t 
know” response if they were unsure of an answer on the 
multiple-choice quiz. Students were not compensated in any 
way for participation and were not informed of either their 
individual performance or the aggregate class performance on 
the quiz. 

RESULTS 

Following the pretest, the multiple-c hoice quiz was scored and 
number of correct, incorrect, and don’t know responses 
recorded for each participant. The mea n and standard deviation 
of correct responses was used to construct five groups of 
students: far below average, below average, average, above 
average, and far above average. These groups correspond to 
response options on the metacognitive instmments. Therange 
of correct responses labeled as average was defined as one 
standard deviation, centered on the mean. Above average was 
defined as the 1.5 standard deviations beyond average; far 
above average as the final two standard deviations. Below 
average and far below average were similarly defined. 
However, due to the low mean value and the large standard 
deviation, there were noobservations in the far below average 
group. There was only one observation in the far above 
average group; this was discarded as an outlier. This left seven 
students in the below average group, eleven in the average 
group, and four in the above average group. Figure 1 shows 
the mean number of correct, incorrect, and don’t know 
responses in each of the three pretest groups. 

Item two on the first metacognitive instrument asked 
participants to rate their “knowledge of concepts and sk ills to 
be covered in the course,” relative to their classmates. 
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Response options were far below average, below average, 
average, above average, and far above average. The 
corresponding item on the second metacognitive instrument 
asked participants to rate “overall performance onthe pretest,” 
again relative to their classmates. Figure 2 shows the nominal 
line representing perfectcalibration and the mean response for 
each pretestgroup on both the first and second metacognitive 



instruments. The distance between the nominal line and the 
actual mean represents the magnitude of miscalibration for the 
group. The average and above average groups were well 
calibrated, though their self-assessments were somewhat 
below their actual performance. As anticipated Ihe below 
average group o verestim ated their performance, though not to 
the degree expected. 



FIGURE 1 

PRETEST QUIZ RESULTS 

n - 22 




FIGURE 2 

MISCALIBRATION AND RECALIBRATION IN THE PRETEST 

n-22 
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At the end of the term, the posttest was administered and 
scores again plotted by pretest group. Overall, the class 
improved their quiz scores by an average oftwelve question s; 
the range was two to twenty. As illustrated in Figure 3, though 
all groups improved their quiz scores, the below average group 
showed the greates t overall i mpro veme nt, an average of fi fteen 
questions. 



What we had hoped to see, of course, was a difference in 
improvement between those who recalibrated on the pretest 
and those who did not. Figure 4 show s mean pretest, posttest, 
and improvement scores, based on correct responses to the 
quiz, grouped by whether recalibration occurred during the 
pretest. Those who recalibrated on the pretest had lower 
pretest scores, on average, than did those who failed to 
recalibrate. However, by the end of the course, there is 
virtually no difference in quizscores between the two groups. 



FIGURE 3 

COMPARISON OF PRETEST AND POSTTEST QUIZ SCORES 

n=22 




FIGURE 4 

COMPARISON OF IMPROVEMENT BY RECALIBRATION GROUP 

n = 22 



Comparison of PreTest Recalibration Groups 
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Figure 5 

Posttcst Quiz Results 

n=22 




Finally, posttest groups were formed in the same manner as 
pretest groups. Again, there were no students in either the far 
below or far above average groups. At the end of the course, 
ten students were below average, six average, and six above 
average. Figure 5 shows the mean number of correct, 
incorrect, and don’t know responses for each posttest group. 
It is interesting to compare Figure 5 to Figure I. Notice that 
the number of incorrect responses for the below average group 
is much higher on the posttest than on the pretest, more than 
double. At thesame time, the number of don’t know responses 
dropped for all groups. 

DISCUSSION 

Are any of these observations statistically significant? 
Unfortunately, errors in data collection resulted in our having 
fewer than half the observations we hoped to acquire over the 
course of the summ er term. As a result, we do not have enough 
data at this time to do the analysis we had planned for this 
conference. We are, however, still collecting data and hope to 
present a more meaningful analysis in the near future. 

Nonetheless, there are some encouraging observations to be 
made. Interestingly, and un expectedly, recalibration seems to 
have occurred as a result of taking the multiple-choice quiz. It 
is difficult to explain why, in the absence of any feedback on 
actual performance, students would recalibrate in such a 
dramatic fashion. In the words of one author, “it looks like we 
scared ’em.” 

To confirm this, an analysis of the pretest data for the Fall 
2000 term was performed to determine if what appears to be 



recalibration is merely a test-retest phenomenon. Because the 
first and second metaco gnitive instruments are not identical., 
the possibility o f a test-retest pro blem see med sm all. However, 
several students in the original study failed to complete the 
second metacognitive instmment, sayingit was a duplicate of 
the first. (Those observations were subsequently discarded.) 
However, given the likelihood that participants perceive these 
two instruments to be the same, itseemed best to consider the 
possibility that what appears to be recalibration is nothing 
more than a test-retest problem. 

Two different sections of the computer literacy course, taught 
by two different faculty m embers, were used for this analysis. 
In one section, the same instrument packet and protocol as 
were used in the Summer 2000 term were used; 3 4 complete 
observations were collected. In the second section, the 
multiple-choice quiz was replaced by a questionnaire asking 
students to provide some background information for the 
instructor. In addition, rather than receiving the second 
metacognitive instrument, which refers to pretest performance, 
these subjects were asked to complete a second copy of the 
first metacognitive instrument. A total of 31 complete 
observations were collected from the second class. 

Recalibration scores were computed as the difference between 
the second self-assessment and the first An ANOVA was 
performed to examine differences between first self- 
assessment, second self-assessment, and recalibration scores 
for the two classes. There w as no significant difference in first 
self-assessment scores for the two groups; p = 0.357. 
However, both the second self-assessment and the 
recalibration means were sign ificantly different; p <0.00 1 . In 
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fact, 16 of the 34 participants who received the multiple- 
choice quiz recalibrated while none of the 31 participants in 
the other class changed their self-assessments. 

The above analysis supports the idea that recalibration is 
occurring simply through exposure to course concepts on the 
multiple-choice quiz. In addition, faculty who have 
administeredthe quiz re port that fewer students approach them 
about “testing out” of the course than in previous semesters. 
Although we canno t quantify this, it does support the idea that 
students adjust their expectations of course content or their 
self-assessm ents of course knowledge as a result of taking the 
quiz. 

Second, either incorrector don’t know responses may provide 
an alternative measure of metacognitive miscalibration, 
perhaps eliminating the need for the metacognitive 
instruments. Looking at Figure 1 with this in mind, the above 
average group is more miscalibrated than the average and 
below average groups on the pretest. While the above average 
students clearly know more than their peers, they still scored 
very poorly on the pretest quiz; they do not know as much as 
they believe they do and, in this sense, are highly 
miscalibrated. By the tim e of the po sttest, when all participan ts 
have been exposed to the same course material., the below 
average group is less well calibrated then the average and 
above average groups (Figure 5). This is consistent with 
claimsthat with dom aincompetence comes theability to make 
more accurate self-assessments; competence begets 
metacognitive (re)calibration . 

Finally, while the study that triggered pursuit of this research 
(Kruger & Dunning, 1999) examined metacognitions at a 
micro level (judgments were solicited for each question), the 
currentstudy takes a more macro approach (a single judgment 
for the entire body of kn owledge). It is encouraging to see the 
same pattern of miscalibration coming from the macro 
approach as was seen in the micro approach. 

CONCLUSIONS AND FUTURE RESEARCH 

It is difficult to draw any meaningful conclusions from such a 
small data set. However, we do believe that instilling domain 
competence is not the only path to metacognitiverecalibration. 
The comprehensive course pretestseems to be producing some 
recalibration, even withoutfeedback on test performance. Only 
additional data, currently being collected, will tell if the 
recalibration is significant and whether recalibration leads to 
improved course performance. 

The results we have see n thus far, c ombin ed with the informal 
observation that the pretest reduces the number of requests to 
“test out” of the course, reinforce our belief that course 
pretests are worthwhile. Furthermore, pretests make 
comparison of pretest and posttest performance possible, 
providing a source of encouragement and a measure of 
effectiveness for the faculty. 



Plans forfurtherstudy include several manipulations including 
providing feedback to students on pretest performance and 
providingcompensation, in the form ofbonus points, for b oth 
pretest and posttest performance. In addition, for the second 
round of data collection adjustments may be made to the 
multiple-choice quiz in order to raise the mean score and 
reduce the standard deviation, in an attempt to populate all five 
student groups. 
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ABSTRACT 

This paper will introduce the knowledge discovery in database (KDD) process applied to the students recruitment 
systems in universities. The definition of the KDD process and its importance will be defined. Different terminology 
for the knowledge discovery process will be discussed with particular emphasis on data mining. The distinction 
between KDD and data mining will be clarified by showing the place of the data mining in the KDD process. The 
tasks and goals data mining are illustrated. The different data mining techniques will be discussed. To place the entire 



KDD process in context, it is applied to a sample data set 
& Technology and Maritime Transport (AASTMT). 

THE EMERGENCE AND DEFINITION 
OF THE KDD PROCESS 

The term KDD was coined in 1989 to point to the 
process of finding knowledge in data. KDD is defined as 
the process of finding patterns hidden information or 
unknown facts in the database (Fayyad, et al., 1996). 
Traditionally the notion of finding useful unknown 
patterns and hidden information in raw data has been 
given many titles including knowledge discovery in 
database, data mining, data archaeology, information 
discovery, knowledge discovery or extraction, and 
information harvesting (Adriaans and Zantinge, 1996). 
The reasons forthis lack of consensus are the novelty of 
the KDD and the multi-disciplinary features of KDD. 
Multi-disciplinary means that KDD belongs to many 
disciplines like statistics and computer (machine 
learning, artificial intelligence (AI), databases, data 
warehousing, expert systems, knowledge acquisition and 
data visualization), from which the KDD process was 



records drawn from the Arab Academy for Science 



drawn(Fayyad, et al., 1996). It is this broad applicability 
that has led number of researchers and scholars to have 
common interest in KDD. 

The interest in KDD has been increased and this is 
demonstrated by the increasing number of forums and 
workshops. Another sources of interest are the various 
publications and special issues that document some of 
the KDD features and foundations (Inmon & Osterfelt 
1991). Although, the field of KDD was founded on 
many disciplines, however, it is gaining its character on 
its own and now stands by itself (Ramakrishnan and 
Grama, 1999). 

KDD OR DATA MINING 

Scientists have usedthetwo terms KDD and data mining 
interchangeably (Ganti, et al., 1999). However, others 
said that data mining is a step in the KDD process 
(Adriaans and Zantinge, 199 6). For the purpose of this 
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paper data mining is considered a step in the KDD 
process. In other words, KDD is an overall process of 
discovering useful knowledge from data, whilst data 
mining points to the application algorithm or technique 
used for extracting patters and unknown information 
from the raw data. So the KDD process will get 
knowledge from the data mining techniquesapplied to a 
certain application. 

THE KDD PROCESS 

The KDD process is interactive, iterative, and involves 
a great deal of user-interference. Brachman and Anand 
( 1 996) defined the practical view of the KDD: 

1. Developing an understanding of the application 
domain. This is an important step because it 
determines and the goals of the KDD application. 
Based on these goals the relevant data mining 
techniques can be employed, where there is no one 
single technique best fits all sources of application 
domains; 

2. Creating a target data set. Selecting the data set, or 
focusing on a subset of the database on which 
discovery will take place; 

3. Data cleaning and preprocessing. Basic operations 
such as the removal of noise if relevant, and 
deciding on the strategy of how to deal with missing 
data items. Example of strategies that might be used 
here are neglecting the incomplete data records or 
setting missing values to null; 

4. Data reduction and projection. Finding useful 
features to represent the data set depending on the 
goal of the task. For examp le if the goal of the KDD 
task is to determine and predict the students’ 
academic performance, not all of the student’s 
record is of importance. Examples are the students’ 
address, telephone number or height and weight; 

5. Choosing the data mining task. This is an important 
step in which the goal of the KDD process is defined 
as weather classification, clustering, summarization 
or others; 

6. Choosing the data mining technique(s) or 
algorithm(s). Selecting the methods to be used for 
searching in the data for patterns and hidden 
information. There should be a match between the 



goal of the KDD and the data mining techniques or 
algorithms; 

7. Data miming. Searching for patterns in the data sets 
using analysis methods and models such as 
regression, clustering, SQL, visualization, decision 
trees and others; 

8. Interpreting the information gained by the mining 
techniques. The output of the mining techniques 
should be evaluated to be understandable and 
consistent. Iterations from steps 1 to 7 may also 
happen when apply KDD to real problem; 

9. Consolidating the discovered knowledge. Reporting 
the knowledge to the interested parties and checking 
the discovered knowledge with the previously 
known knowledge. 

During the KDD process, particularly the data mining 
step, it is necessary to search the database of the 
organisation. When the organisation database(s) contain 
history of data for years it is said to be data warehouse 
(DW). When the DW contains all organisational data it 
is called enterprise DW, whilst if the DW contains 
functional data about marketing, personnel, or 
production it is said to be a data mart. The data 
warehouse (DW) will enhance the KDD results (Taha, 
et. al., 1997; Berson and Smith, 1997; Barquin, 1997; 
Paller, 1 997). So, data warehouse will enhance the KDD 
process through the wealth of historical data it offers to 
the mining techniques (Berson and Smith, 1997; 
Adriaans and Zanting, 1996). thediscovered knowledge 
should be utilized in a suitable front-end tool like 
executive information systems(EIS)or decision support 
systems (DSS). The following Figure 1 describes the 
whole KDD process. 

THE PRIMARY TASKS OF DATA MINING 

The two high-level fundamental goals of data mining in 
practice are prediction and description (Fayyad, et al., 
1996). Prediction is the use of some variables or data 
fields in the database to predict the unknown future 
values of the other variables or data fields of interest. 
Description focuses on finding human understandable 
patterns describing thedata set. Therelative necessity of 
both of these goals varies from application to another. 
Both goal prediction and description are achieved using 
the following data mining tasks. 
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FIGURE 1 

THE KDD PROCESS OVERVIEW 




Or DW 
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data 
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1 . Classification. It is a learning function that classifies 
a data into one of several predefined classes. 
Example is found in classifying students concerning 
the scholarship the university grants into granted and 
not-granted, based on GPA (grade point average); 

2. Regression. A regression model is a mathematical 
equation that provides predictions of the values of 
one variable (dependent) based on the known values 
of one or more other variables (independent). If one 
predictor is considered then the regression is called 
simple linearregression, if more then the regression 
is called multiple linear regression (Canavos and 
Miller, 1995); 

3. Clustering. Clustering is the process of producing 
classifications from initially unclassified data 
(Everitt, 1981). It is a common descriptive task 
where one seeks to identify a finite set of categories 
or clusters to describe the data; 

4. Summarization. The process of finding a compact 
description for a subset of data. Examples of 
summarization like finding the mean and standard 
deviation of all data fields of interest, or the 
discovery of the relationships between variables; 

5. Change and deviation detection. It focuses on 
discovering the most significant changes in the data 
from previously measured or normative values. 



DISCUSSION OF COMMON 
DATA MINING TECHNIQUES 

Any form that would help extracting more patterns and 
hidden information from the database is called a data 
mining technique (Adriaans and Zantinge, 1 996), Many 
classifications are found for data mining techniques 
based on many variant factors of classification 
(Ramakrishnan and Grama, 1999), like: 

1. The induced representation (decision trees, rules, 
correlations, deviations, trends, and associations); 

2. The data they operate on (time series, discrete, 
labelled, continuous, or nominal); 

3. The application domains (finance, economic, 
biology, Web log mining and the like). 

Another said that the data mining techniques are many 
(Adriaans and Zantinge, 1996), these are; Query tools. 
Visualization, On-line analytical processing (OLAP), 
Association rules, Case-based learning (nearest 
neighbour), Decision trees. Statistical techniques. 
Genetic algorithms (GA), andArtificialneural networks 
(ANN). Discussions of the techniques thatare ofinterest 
to this research are developed in the following sections. 
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Query Tools 

Traditional query tools are first used to analyze the data 
sets. By applying simple structured query language 
(SQL) we can obtain a wealth of information. However, 
before we can apply more advanced pattern analysis 
algorithms, we need to know some basic aspects and 
structuresofthe data set. With S QL we can unco ver only 
shallow knowledge that is easily accessible from the data 
set: yet although we cannot find hidden knowledge. 
Adriaans and Zantinge (1996) said, 

for the most part 80% of the interesting 
information can be abstracted from a database 
using SQL. The remaining 20% of hidden 
requires more advanced techniques, 

Howeve r, for most organizations this 20% of the hidden 
knowledge have 80% of the importance in relation to 
decision making, and the 80% information volume 
represent only 20% in terms of value to the decision 
making process. A good way to start is to extract some 
simple statistical information fi’om the data using SQL 
queries. For example: 

How many students in the university taking the 
accounting major? 

What is the average GPA for the male students with 
an American diploma background? 

How many grants go to junior students? 

What is the nationality distribution of students? 

Decisionscquld be takenbased on the output ofthe SQL 
statements. 

Visualization 

Visualization technique depends strongly on the human 
side of the analysis (Berson, 1996). Even the best set of 
rules or tables of data may reveal more information 
when visualized with color, relief, ortexture in 2Z), SD 
and even 4D representations. In the 4D, 3D are mapped 
onto the screen and the fourth can be expressed through 
the use of color (Berson and Smith, 1997). Visualization 
technique may be used throughout the data exploration 
process and are particularly useful during the initial 
stages of the high-level groupings of data sets. An 
elementary technique that can be of great value is the 
scatter diagram', in which information on two attributes 



is displayed in a Cartesian space. Scatter diagrams can 
be used to identify interesting sub-sets ofthe data sets so 
that we can focus on the rest of the data mining process 
(Adriaans & Zantinge, 1996). 

Data visualization is emerging as an advanced 
technology that may allow organizations to process 
amounts of information and present it in a usable format. 
Through the interface visualization techniques provide 
the non-computer users to navigate through data using 
their human feelings of the data displayed. Visualization 
technique plays a great role, that is it puts the 
information we have in an easy and understandable way 
for both computer and non-computer aware people 
(Keim, et al., 1996). The following benefits are earned 
beyond the use of visualization techniques (Berson, 
1996): 

Users can easily in teract with attributes and illustrate 
how they affect certain phenomenon; 

Users can view summarized data with drill down 
capability; 

Find hidden patterns; 

It is considered an exploratory data analysis tool 
(EDA). That is data navigation, comparison, scaling, 
filtering are available to users. 

On Line Analytical Processing (OLAP) Tools 

The need for OLAP was developed to handle situations 
where the relational database management systems 
(RDBMS) can not deal with the multidimensional 
problems. Although RDBMS are powerful solutions for 
a wide range of commercial and scientific applications, 
they are not good at addressing the modern business 
analysis, forecasting, and all the like that are 
multidimensional in nature (Berson, 1996). The key 
driver for OLAP is the multi-dimensional nature of the 
issues it deals with. For example managers might ask 
question like the following: Describe the relationship 
between majors, nationalities, ages, and GPA? 

OLAP is a continuos and iterati veprocess, an analyst may 
drill down to see much more details. And this discovery of 
knowledge forces the managers to ask more complex 
questions. Multidim ensionality is the core of a number of 
OLAP systems available. OLAP involves several basic 
analytical operations including consolidation, drill-down, 
and statistical techniques (O’Brien, 1996). 
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FIGURE 2 
OLAP EXAMPLE 
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1. Consolidation. Consolidation involves the 
aggregation of data, i.e. the total number of students 
at the university, total courses, and average GPA; 

2. Drill-down. This is the opposite ofconsolidation that 
involves more details; 

3. Slicing and dicing. Slicing and dicing refers to the 
ability to look at the database from different 
viewpoints. 

Association Rules 

Association rules are alw ays define d on bin ary attributes. 
This sort of attributes makes it easy to describe the 
student profiles in our example database. Example is: 
Male student, Nautical department^ Egyptian 
nationality. This means that if the student ’s gender is 
male and the major is nautical, so the student’s 
nationality is Egyptian. And this happening 65% 
confidence level. 

In fact, the number on possible association rules that 
might be found in a database is almost infinite. Some 
roles are found to be useless (Argawal, et al., 1 996). The 
level of Confidence is a mechanism used eliminates 
useless rules, with low confidence levels. Confiden ce is 
the percentage of records that holds true for the fact 
under studying within the group of records. 

Decision Trees and Rules 

A decision tree is a predictive model that can be viewed 
as a tree. Each branch of the tree is a classification 
question, and the leaves are p artitions of the data set with 



their classification (Berson and Smith, 1997). Another 
definition for the decision tree uses logical methods of 
describingregions of state. These logical methods could 
be interpretedin a‘TF..THEN”rules space (Pyle, 1999). 
One variable is studied individually in a decision tree. 
The start point is found when the variable that best 
classifies the state space is determined and consequently 
the true state is also raised. The algorithm of the tree 
then looks for another classifying variable and another 
splitting rule. This process continues until some ending 
criteria is found. Another algorithm for developing the 
decision tree is called ID3 is iterative algorithm. TheID3 
start with a subset of the data called window. The 
window is chosen at random and then develops the tree 
which is correctly classifies the subset into branches. 
Then all other data are classified using the tree. If each 
data record finds its classification so the process 
terminates. Ifnot, a selection of the incorrectly classified 
data records is added to the window and the process 
continues (Quinlan, 1986). 

The decision tree is a useful technique in both data 
mining and predictive modelling processes. It prevents 
the problems of overfitting (it happens when the 
algorithm searches in the limited data sets, so the 
algorithm might overfit the data) and handling of 
missing data that majority of the mining techniques 
always leave these problem in the user side. The decision 
tree classifies the data into branches without loosing any 
of the data records. 

Artificial Neural Networks (ANN) 

The term artificial neural netw orks (ANN) is a computer 
programs that implements complex pattern detection and 
machine learning algorithms to build predictive models 
from large database(s). In order for the ANN to detect 
patterns in the data sets, it should learn to detect these 
patterns and make predictions, is the same like a human 
being does. ANN are widely used in many business 
applications. Most of the ANN are hard for users to 
understand because they lack clarity, however, vendors 
tend to introduce the ANN in visualization formats that 
make them understandable. Also an ANN is a time 
consuming process to build, however, it is a very 
powerful predictive technique that require some data 
preprocessing, good understanding of the problem and 
the target of prediction. It also requires the setting of 
some parameters that will drive its mission (Berson and 
Smith, 1997). 
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Nearest Neighbour 

Nearest neighbour is a prediction technique that is used 
to perform data clustering. Many other techniques are 
used for clustering like the complete linkage method or 
the furthest neighbour. Centroid cluster ana lysis , Media 
analysis, Group average method, and automatic 
interaction detection (Everitt, 1981). Since they all 
perform thesamejob, which is data classification or data 
clustering, one of them is illustrated here. 

It begins with a distance matrix that shows the individual 
groups and end with a dendrogram showing the 
successive fusions which culminates all individuals in 
one group (Everitt, 1981). Groups of single individuals 
are fused with their nearest members. Each fusion 
decreases by one the number of groups. The concept of 
nearest neighbour states that records that are close to 
each other live in each other's neighbour. For example, 
if we need to predict the performance of a student we 
first look at the students that are lose to him in the 
database. 

Nearest neighbour is usedinmany business applications 
like bankruptcy prediction in the banking industry and 
handwriting recognition. The nearest neighbour 
technique is automated but require some preprocessing 
of data in converting some predictions into values that 
will be used to measure distance. However, unordered 
variables like eye-color should be transformed into the 
distance between each other when there is a match found 
(Berson and Smith, 1 997). The techniques will give the 
user the high-levelview of what is going in the database. 
The nearest neighbouris optimized for prediction of new 
records rather than exhaustive extraction of interesting 
rules from the database. Most of the text retrieval 
systems are built around the nearest neighbour 
technologies. A problem in the nearest neighbouris the 
existence of tables consists of high number of attributes. 
Nearest neighbour does not perform well if there is a 
very big number of independent attributes because this 
will need a multi-dimensional search space. Possible 
problems associated aremillions ofvery complex and it 
is also possible to find an equal space between data that 
is hard to define the nearest of them. One approach here 
is to limit thenumber of attributes by finding the relative 
importance of them and work on them, or to search for 
another technique that is able to handle the large number 
of attributes. 



Genetic Algorithms 

This term is a combination ofboth biology and computer 
disciplines, and sometimes referred to as simulated 
evolution. Berson and Smith (1997) said “Genetic 
algorithms loosely refer to these simulated evolutionary 
systems, but more precisely these are the algorithms that 
dictate how populations of organisms should be formed, 
evaluated and modified”. Genetic a Igorithm s are used to 
create the biological evolution version of computers. 
They start on a small program and then mature 
themselves as the human organisms undergoing natural 
evolution. Over time, these programs on the computer 
improve in their performance and as a resultincrease the 
efficiency of resolving a certainproblem. In many ways 
genetic algorithms are close to the biological evolution, 
the analogy is like Table 1 . 

Genetic algorithms are used to find optimal clusters 
based on a defined profit measure. They have been used 
to optimize the nearest neighbour classification systems 
for predicting sequences in time series (Berson and 
Smith, 1997). 

Probabilistic Graphical Dependency Technique 

These models specify the probabilistic dependencies, 
which underlie a particular model using a graphical 
structure. The model specifies which variables are 
dependent on each other. These models are used for 
categorical or discrete-valued variables, however, some 
extensions allowed th e use for rea 1-valued variables also. 
Within artificial intelligence and statistics these models 
are initially built in the context of the probabilistic expert 
systems (ES). Although, graphical models induction is 
still not a mature discipline, it is of interest to the KDD 
applications since graphical forms of the model are 
easily understood by users (Fayyad et al., 1996). 

EXAMPLE FOR THE KDD PROCESS APPLIED 
TO STUDENT RECRUITMENT SYSTEMS 

The KDD process will be applied to a sample data 
records (1600 records) extracted from the students’ 
recruitment system database used at AASTMT.* Not all 
of the database fields were used for this purpose; rather 
the data, which are of importance to the executive 
managers that, are entailed in the students’ recruitment 
system functions. 
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TABLE 1 

GENETIC ALGORITHMS AND BIOLOGY 



Biology 


Genetic Algorithms 


-Organism 


Which is the computer program being optimized 


-Population 


The collection of programs undergoing simulated evolution 


-Chromosome 


The chromosome encodes the computer program 


-Fitness 


The calculation with which a program value is determined for 
survival ofthe fittest 


-Gene 


The basic building block of the chromosome that defines one 
particular feature of the simulated organism 


-Locus 


The location of the chromosome that contains a specific gene 


-Allele 


The value of the gene 


-Mutation 


The random change of the value of the gene 


-Mating 


The process by which two simulated programs swap pieces of 
them in a simulated crossover 


-Selection 


The best program is retained and the less successful are excluded 
by deleting them from computer memory. 



Data Selection 

In the sample under study, 1600 records of the 
applicants, from which a number of 1100 has been 
accepted at AASTMT (AASTMT application records, 
1995; AASTMT statistics, 1995). The records consistof 
serial number, application number, first name, 
nationality, sex, address, desire(s), percentage grade, 
accepted/ rejected, and department. In order to facilitate 
the KDD process a copy of this operational data was 
drawn and stored separately, sample records of this 
database is given here in Table 2. 

Cleaning 

Several methods are available to clean the data i.e. 
remove errors. Some of these methods can be executed 
in advance wh ile others are only invoked aftererrors are 
detected at the coding or the discovery stage (Adriaans 
& Zantinge,1996). A very important element in a 
cleaning operation is the de^duplication of records 
(Table 3). In the student database file one student may be 
presented by more than one record. For example, if a 
student applied from abroad and then a relative also 
applied on behalf of the student, this should be 
positioned another application for the same person, this 
would be clear from the check ofdata. Another source of 
error is that students change their address without 



notifyingthe Admission & Registration office. There are 
also cases in which people spell their names incorrectly 
or give incorrect information about themselves by 
slightly misspelling their name or by giving a false 
address. Data cleaning processes affect thequality ofthe 
mining process, because as much seriously the process 
of data cleaning was performed, the results of data 
mining would be helpful and trustworthy. 

In thepresent example we havetwo different records for 
the same student data, so this may be due to two persons 
submitted the same student data without the student 
being aware of that. Of course, we can never be sure of 
this, but de-duplication algorithm using analysis 
techniques could identity the situation and present it to 
a user to make a decision. The second type ofdata errors 
that frequently occurs is the lack of domain consistency, 
(Table 4). 

In Table 4 the first student department is empty, 
however, this attribute should have a value. Empty 
values are a source of problems to the data mining 
process, because this might affect the type of patters 
discovered. If the data item is not defined it should be 
NULL. On the second record there is inconsistency in 
the domain of nationalities and cities, the opposite is 
true; replacement would happen between city, and 
nation. The result is shown in Table 5. 
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Enrichment 

Assume that we have got some extra information about 
the students’ family annual income, and the student 
secondary school, where it is possible to get or buying 
them. See Table 6. 

Coding (Pre-coded data) 

In the nextstage, we select only those records that have 
enough information to be of value. Notice that the extra 
information should be added to the original data. See 
Table 7. 

A general rule states that any deletion of data must be a 
conscious decision, after a thorough analysis of the 
possible consequences. However, in some cases lack of 
information can be a valuable indication of interesting 
patterns. In the presented cases in Table 7 for the 
students Tarek, and Wael, we lack some vital data 
concerning them, so we choose to exclude their records 
from the final sample. Of course, this decision is 
questionable, because there may be a causal connection; 
however, it is better to delete incomplete information 
instead of getting incorrect results (Adriaans and 
Zantnige, 1996). Next we carry out a projection of the 
records. In our example we are not interested in the 
students’ name, so their names are removed from the 
sample. Up to this point, the coding phase consisted of 
nothing more than simple SQL operations but now we 
are entering the stage where we will be able to perform 
some data transformations. 

Coding (Post-coding Data) 

Coding is a creative activity that has to be performed 
repeatedly in order to get the best results. Income even 
is a source of problem. One solution might be to 
transform income into categories and describe each 
category’s characteristics. For example: annual income 
between 300.000-200.000 L.E is called moderate-and 
coded 01, and annual income between 1,000,000- 
2,000,000 L.E is called premium income-and coded 05. 
So thatwe can obtain information such as thefollowing: 



1. Students applied from nationality Egyptian, in the 
Maritime department, have a premium income 
group. OR 

2. Students from Syria, always have Engineering as a 
major, with average marks of 60%. 

Instead of doing a studentdata analysis, the relationships 
between students of diffe rent nationa lities are imp ortant. 
This means that we will not be investigating the 
connections between individual attributes, but between 
different student profiles. Before going in depth to the 
results, a complete coding process should take place. 

Table 8 represents the new table that results from the 
coding process. A table in this format isnot very helpful 
ifwe need to find relationships between different student 
profiles. Each studentis represented by one record. That 
is, instead of having one attribute with five different 
possible values, we create twe nty binary (0 and 1 are the 
components of the code) attributes, one for each 
nationality. If the value of the nationality attribute is 
“10” this means that the student is Egyptian, 20 means 
Sudanese, 30 means Libyan etc. Such an operation is 
called {Flattening) an attribute with cardinality n is 
replaced by n binary attributes. Applying the concept 
Flattening to the Sex results in 01, 02, which means that 
male is replaced by 01, and female is replaced by 02 
{cardinality 2). 

Data Mining Techniques 

The data mining techniques were discussed with respect 
to many classification mechanisms, however, the KDD 
example here will not definitely handle all of the 
techniques because there is no single application that 
uses all of the data mining techniques, and that is due to 
the differencesbetween .techniques and data sets and the 
application domains. 

Traditional query tools 

A good way to start is to extract some simple statistical 
information from the data set. Table 9 provides statistics 
on students. 
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TABLE 2 

SAMPLE OF THE ORIGINAL DATA 



SN 


App. No 


F .N a m e 


Sex 


Nation 


City 


Desire 


% mark 


A/R 


Dept 


1 


697 


Mary 


F 


Egy 


Alex 


Hot 


95 


A 


Hot 


4 


1079 


Asser 


M 


Egy 


Alex 


Bus 


70 


A 


Hot 


197 


1484 


Ismael 


M 


Syr 


Dam 


Eng 


61 


A 


Eng 


1 


1570 


Mohamed 


M 


Egy 


Alex 


Hot 


90 


R 




4 


19 


Lamees 


F 


Pal 


Dub 


Bus 


83 


R 





Note: SN stands for serial number, App.No for application number, F.Name for first name, Nation for nationality, 

A for accepted & Rfor rejected, and dept for department. Also, the departments, and nationality names have 
been curtailed for space purpose in the table. 

TABLE 3 

DE-DUPLICATION OF RECORDS 



SN 


App. No 


Name 


Sex 


Nation 


City 


Desire 


% mark 


A/R 


Dept 


270 


316 


Ehab 


M 


Egy 


Alex 


Eng 


70 


A 


Eng 


271 


623 


Ehab 


M 


Egy 


Alex 


Eng 


70 


A 


Eng 



TABLE 4 

DOMAIN CONSISTENCY 



SN 


App. No 


Name 


Sex 


Nation 


City 


Desire 


% mark 


A/R 


Dept 


541 


350 


Ahmed 


M 


Egy 


Cairo 


Eng 


70 


A 




215 


476 


Ahmed 


M 


Alex 


Lyb 


Eng 


70 


A 


Eng 



TABLE 5 

DOMAIN CONSISTENCY-1 



SN 


App. No 


Name 


Sex 


Nation 


City 


Desire 


% mark 


A/R 


Dept 


541 


350 


Ahmed 


M 


Egy 


Cairo 


Eng 


70 


A 


NULL 


215 


476 


Ahmed 


M 


Lyb 


Alex 


Eng 


70 


A 


Eng 



TABLE 6 
ENRICHMENT 



SN 


App. No 


Name 


I n CO m e 


School 


Nation 


Dept 


100 


801 


Ola 


200.000 


Saudi Sch. 


Sau 


NULL 


200 


802 


Alaa 


350000 


IGCSE 


Leb 


Eng 



TABLE 7 

ENRICHED TABLE 



SN 


App. No 


Name 


Inc. 


Car 


Sex 


Nation 


City 


Desire 


% 


A/R 


Dept 


911 


350 


Tarek 


Null 


Null 


M 


Egy 


Cairo 


Eng 


60 


A 


Null 


111 


714 


Wael 


150 


Null 


M 


Null 


Alex 


Mar 


85 


A 


Eng 
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TABLE 8 

THE CODING EFFECT 



SN 


App. No 


Inc. 


Car 


Sex 


Nation 


City 


Desire 


% 


A/R 


Dept 


911 


350 


05 


Null 


M 


10 


101 


1 


60 


A 


1 


111 


714 


01 


Null 


M 


30 


301 


4 


85 


A 


4 



TABLE 9 
STATISTICS 



Department 


Private* 


Sponsor** 


Transfer 


Total 


Nautical 


54 


6 


0 


60 


Maritime Eng. 


37 


2 


0 


39 


Mechanical Eng. 


51 


0 


2 


53 


Computer Eng. 


93 


6 


6 


105 


Power Eng. 


64 


0 


0 


64 


Electronics Eng. 


78 


5 


5 


88 


Construction Eng. 


61 


0 


0 


61 


Managerial Eng. 


58 


0 


0 


58 


Business Adm. 


181 


6 


7 


194 


Hotels & tourism 


27 


0 


1 


28 


Grand total 


704 


25 


21 


750 



* Private private student means that he/she pays for himself/herself. 

** Sponsor sponsored student appears when a certain agency pays for the student fees and 
accommodation, regardless ofthe reason. 



A number of 350 students should be added to the grand 
total of the previous table”! 100 total accepted”, they 
represent the number of students that will join the 
preparatory program so that, after succeeding they can 
join the next semester unless rejected. From table (9), 
and the originaldata sets, a lot of shallow knowledge can 
be extracted and represented. For examples, the 
percentage of sponsored students at the AASTMT- 
according to the sample data records- is about 3%, as 
well as the transfer students “i/ the AASTMT has 1000 
students 940 are predicted to be private, 30 are transfer, 
and 30 are sponsored'. The analysis is not a goal in 
itself, rather the implications of the analysis to the 
executive who will take a decision according to that 
analysis. For example, the drill down capability reveals 
that the 3% of students who are sponsored historically 
joinedtheNauticaldepartment whichhas a little demand 
nowadays (AASTMT statistics, 1 990-1997). May be the 
reason is the student is actually sponsored but he claims 



the opposite to save money, this is because sponsored 
students pay higher fees. Also, the factthat the Business 
Administration has the highest demand may be traced 
and mined as follows: 

Maybe the competitor universities are weak in this 
field. 

Maybe the excellent facilities the AASTMT has 
made this possible. 

Mayte the staff members of the Business 
Administration departm ent are sup erior to its rivals, 
and use nice textbooks. 

No matter what the reason is the analytical ability offer 
the different possibilities to the executives and find the 
hidden knowledge and patterns, then the role of the 
executive is to take corrective actions. 
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Visualization Techniques 

In Figure 3, visualization technique plays a great role, 
that is it puts the information we have in an easy to 
understand way. Table 9 has been transformed to the 
Figure 3. 

OLAP 

We can plot Table 9 in the following format in Figure 5. 
The figure represents the students’ data against 
departments, regarding if they are sponsored, private, or 
transfer students, but in a category of details level. 



Association Rules 

For the purpose of our database set we can develop a set 
of rules. For example: 93% of the students taking the 
Bus. Adm. major “^25% of total students. The 
previous rule importance is that, what is applied to the 
private students at the Business Administration 
department is applied to 25% of the total private 
students. 

Decision Trees and Rules 

A decision tree like the following can be found in the 
admission and registration function. 



FIGURE 3 

VISUALIZING NEW KNOWLEDGE 




□ 150-200 

□ 100-150 
■ 50-100 

□ 0-50 



FIGURE 4 

SLICING AND DICING 




S3 



□ Seties 1 
■ Seues2 
n Setiies3 
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FIGURE 5 

DECISION TREE FOR STUDENTS’GRANTS 
Y N 




- The example follows the credit hours system at the AASTMT. 

Rule 1: IF GPA is less than 1 .6 THEN student is on probation. 

Rule 2: IF 1 .6 is less than or equal GPA is less than 2THEN student has restricted load. 
Rule 3: IF 2 is less than or equal GPA is less than 3.6 THEN student takes award. 

Rule 4: IF 3.6 is less than or equal GPA is less than or equal 4 THEN student takes grant. 

Nearest Neighbour 

Assume that we have 4 student GPA groups. As in the following matrix: 



FIGURE 6 
DENDROGRAM 



Reporting 



1 


1 

0 


2 

3.2 


3 

3.4 


4 

3.8 


Matrix G 1= 2 


3.2 


0 


4 


2 


3 


3.4 


4 


0 


1.9 


4 


L- 3.8 


2 


1.9 


0 -J 



Sicp (I): The smallest emt)'is 034, so group 3 and 4 are fused logdlicr. Distance between G34 and 
groups I and 2 ; 

G(34)Umin{G31,G41)=G31=3.4 
G(34)2 =min |G32.G42}=G42=2 
Step (2): Mairis G2 is the following, 

1 2 (34) 

1 0 3.2 3.4 

Matrix G2= 2 3.2 0 2 

(34)L 3.4 2 0 J 

Ihe smallest entry is G2(34). so group 2 and (34) are fused together. Distance between G2(34) and 
group I: 

G(34) Urain (G31,G4I }=G31 =3.4 
1 2(34) 

Matrix G3= 1 ^ 3.1^ 

2(34)L3.2 0 J 



The dendrogram is the following. 



3.2 




Reporting the results of data mining can take many 
forms. In general, one can use any report writer or 
graphical tool to make the results of the process 
accessible (Adriaans and Zantinge, 1996). For some 
applications, like the students’ recruitment system the 
reports might be connected to an EIS ora DSS (Taha, 
et.al, 1997). It has been found that the interactive 
character of the KDD process leads to the consistent 
interplay between the selection of data, cleaning, data 
mining, and the reporting results. 

KDD FUTURE RESEARCH 

larger databases/algorithm scalability and 
dimensionality. The mining techniques as step in the 
KDD process should be able to handle database with 
hundreds of tables and fields, as well as with 
millions of records. This may cause induction and 
time problems. 
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overfitting. This happens when the mining algorithm 
searches for the best parameter for one model using 
a limited data set. 

changing data and knowledge. Rapidly changing 
data may cause the discovered patterns to be 
irrelevantand spurious. Example is the stockmarket 
data. 

missing and noisy data. Here the problem emerges 
when important data items are missing. This 
problem often occurs when the database design is 
carried out without taking into consideration the 
discovery process. 

understandability of discovered knowledge. One 
critical factor in KDD applications is to make the 
output of the systems human-understandable. 

data security. This is related to the security of the 
original data and how should the mining techniques 
deal with this data without violating their security. 

CONCLUSION 

KDD istheprocessoffininghidden knowledge, patterns 
and unknown facts from the data sets. KDD is an entire 
process starting with selecting the application domain 
ending with the output knowledge, which needs a 
suitable front-end tool. The front-end might be EIS, or 
DSS. Data mining is a step in the KDD process. The 
output knowledge of the KDD process will enhance the 
decision quality. The data mining techniques can be 
applied to the students’ recruitment systems at 
universities. 

ENDNOTE 

1 . Data used with permission. 
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USING HEURISTIC 
SELF-STUDY TO RESEARCH 
INFORMATION SYSTEMS EDUCATION 



Mark Campbell Williams 
Edith Cowan University 



ABSTRACT 

Using a conversational writing style, I report on a heuristic inquiry research approach used in a Jive year project 
investigating technicism and discourse in a University Business Computing teaching reform. Briefly describing the 
research itself I concentrate on exp laining the nature of heuristic self-study which has both strengthsand weaknesses 
for educational research. To exemplify the heuristic idea of self-dialogue, the paper is in the form of a fictional 
dialogue. 



INTRODUCTION 

From 1991 to 1995, 1 investigated a teaching reform of 
a University Information Systems Course involvingthree 
lecturers, eleven tutors, and hundreds of students. The 
essence of this reform was to balance a prevailing 
technicism by encouraging open discourse through the 
introduction of teaching and learning strategies such as 
group work, a dialogical communication process in the 
tutorials and mass lectures, and requiring students to 
keep personal learning journals. For example, at the 
initial computing laboratories, the tutor and students 
would share their name, computing experience and 
general educational and work experience. In following 
laboratories, the tutor would ask students to mention the 
ways in which they were making progress in learning, 
and perhaps applying, the information systems concepts 
and practice in their everyday life. The following 
quotation from an the Course Unit Outline (Semester 
One, 1993) describes the format of the learning journal: 

The student general learning journals may include some 
or all of the following headings: 

a. A significant personal learning 
experience a powerful learning highlight 
(what you learnt, and how you learnt it) that 
was personally relevant to either your life 



and/or work. This would not usually be a 
formal or conceptual insight but rather an 
existential experience to do with your 
personal journey of discovery in this area. 
You should also include insights or 
experiences made during the week either at 
work or at home. We would be especially 
interested in any insight you may have 
regarding the way in which this unit may be 
part of your overall intellectual and moral 
development and also in comments on the 
affective area (feelings or emotions). 

b. Reflective examination of your own 
performance and contributions in lectures 
and tutorials, and your participation in the 
unit. A record of your comments or 
questions made in lectures or tutorials 
should be noted as should, perhaps more 
importantly, any relevant informal 
discussions or interactions either with the 
learning group or outside. You could also 
comment on your perceptions of the 
personal and communal group dynamics in 
the tutorials (for example, who is doing 
what and why; if and how the learning 
group is helpful to the learning process. 
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formal and informal leaders, co-operation, 
etc.) 

c. Every second week you should include 
newspaper and journal article clippings that 
have contributed to your learning 
experience. 

In an introduction to the journal you should list 
your beginning competency level and previous 
experience (approx, one page) and your 
expectations for the course (approx, one page). 

A reflectivesummary (approx, one page) and an 
account of your own values and goals with 
relevance to information technology (approx, 
one page) should conclude the journal 
(Information Systems Course Outline, second 
semester, 1993, p. 4). 

As well as discussing points raised in their journals with 
individual students, I also discussed interesting general 
points with groups of students. I used interpersonal 
communication skills such as active and passive 
listening and T language (Gordon, 1976) to encourage 
students to ask and answer questions. I forcedmyself to 
wait in silence until a reply came from the group and 
then to wait for several seconds before replying or 
responding to any student comment. 

For two years, I collected qualitative research data 
concerning the impact of the teaching reform. However, 
through a component of self-study in the research, I 
came to understand that I, myself, had succumbed to a 
form of inner technicism in my conduct of the research. 
I had imposed the teaching strategies on the students and 
tutors, not gaining their full permission or imparting 
understanding of the research, in my drive to 
gain a research result. I was thus led to conduct an 
heuristic inquiry with the question "what was the 
underlyingreasonsforthe ethical shortcomings, and how 
could I address this problem?". 

OPEN DISCOURSE AND TECHNICISM 

Open discourse is communication concerning personal 
and social meaning, purpose, orientation, values, goals, 
concepts, ideas, feelings and emotions relevant to, but 
distinct from, the narrow subject matter, technique and 
administration of University Business Computing 
education. In a University Business Computing tutorial, 
for example, open discourse could occur in the initial 
introductions of the participants or in initial statements 



of the meaning and purpose of the subject to students' 
life situations or goals. Any discourse specifically 
concerned with matters of computing techniques or the 
terminology of the text would not be open discourse 
(although open discourse may occur if the discussion 
broadensto include personal, historical or societal issues 
relating to the meaning or purpose of the techniques). 

Open discourse does not necessarily have to be formal 
small group or whole group discussion. It may even be 
a passing comm ent to an individual student afterthe end 
of the computing laboratory, or an informal chat to a 
group of students in the campus grounds, or even a note 
on assessing a student's work (written communication 
can also be discourse). The teacher and students can be 
autonomous, free, well-rounded human beings able to 
engage in meaningful and balanced discourse even in 
spite of a technicist curriculum (Taylor and Campbell 
Williams, 1993b). 

Technicism is an over-emphasis on technical, 
instrumental or strategic techniques or actions, to the 
detriment of wider human communication, human values 
or human purpose (Adorno & Horkheimer, 1990; 
Dryzek, 1990; Ellul, 1990; Habermas, ^1972; Marcuse, 
1 964, 1 969). Persons thinking in technicist ways would 
tend to define all human problems in terms of rational 
and technical solutions, thus leading to an undue 
emphasis on the science, instrumental rationality and 
technology of the modern age (Bowers, 1988, 1993 a, 
1993 b; Ellul, 1964, 197 3; Habermas, 1971). 

A DIALOGUE 

In this paper, written in the form of a dialogical style, I 
report on the research, emphasising the opportunities and 
challenges afforded by heuristic research. As part of the 
self dialogue (Moust£LksiS, 1990) component of heuristic 
research I conducted the following inner dialogue with 
myself, with a figure I named Auguste Comte acting as 
a devil's advocate. 

Comte: If the 1991 to 1993 qualitative study was 
flawed, why didn't you just discard it and begin a new 
investigation? 

Mark: Professor C . S. Lewis's (1 943) writes: 

A wrong sum can be put right: but only by 
going back till you find the errorand working it 
afresh from that point, never by simply going 
on, Evil can be undone, but it cannot "develop" 
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into good. Time does not heal it. The spell 
must be un-wound, bit by bit, "with 
backward mutters of dissevering 
power" or else not (p. 9). 

If I did not unwind the 'spell' that caused me to ruin the 
first empirical study, could it not return to ru in a second 
attempt? So, "with backward mutters of dissevering 
power" (p. 6), I decided to investigate my ethical 
shortcomings. To explore my hunch that I had 
succumbed to an inner technicism, I entered a new phase 
of the research by employing what Moustakas (1990) 
terms heuristic inquiry to reflect on "what was the reason 
underlying my ethical shortcomings, and how could I 
address this." 

I thought it legitimate to heuristically analysis some of 
the research data from my flawed 1991 to 1993 
qualitative study. The impressions thus gained about 
technicism and open discourse in the outer educational 
context of the computing classes could act as a backdrop 
for the heuristic self-study of what I term my self-as- 
researcher. The impressions were: (1) that non- 
interaction in classes was an indication oftechnicism; (2) 
that technicism was encouraged by the nature of the 
tutorial work-book and the way it was used by the tutors; 
and (3) that I had unethically operated from an inner 
technicism in unethically imposing the research on the 
students and other teachers. 

I use Lenzo's (1 995) term "researcher-as-self to allude 
to the way in which "I" was involved in the research. I 
use thetemi true self, or Self, to referto that fundamental 
and central inner energy of a person which actualises the 
wholeness of the psyche and is the culmination of 
personal development (Jung, 1968, p. 199). 

Comte: Let's begin with the study, before you launch 
into metaphysics. What labels wouldyou use to describe 
your research? 

Mark: The study could be labelled using various 
categorisations. I conducted an heuristic inquiry re- 
examining the research material that I had collected in 
the 1991 to 1993 qualitative study which I discontinued. 
Building on one of the impressionsi gained, I conducted 
a psychologically-based heuristic reflection from 1993 
to 1996. Accordingto Lancy's typology (1 993, pp.7, 1 1, 
15, 183), my study can be said to be an independent, 
teacher-researcher, personal account. Personal account 
(Lancy, 1 993, p. 7) can be autobiography, biography or 
oral history employing long-term interview, diary, 



journal and content analysis with typical foci being the 
process of becoming a teacher and the relation of 
teaching to other aspects of the life cycle. As an 
independent study (Lancy, 1993, p. 11), my study 
investigates issues that would not be likely to attractthe 
interest and attention of a quantitative researcher. A 
teacher-researcher (Lancy, 1993, p. 15) research study 
implies that the role of the investigatoris that ofteacher 
and that of researcher. 

Lancy (1993) points out that a research written as an 
autobiography does not need to generate assertions or 
make judgements on the material. As Lancy comments, 
I am comfortable with the suggestion that "there is no 
single interpretation. Each reader must deconstruct . . . 
the texts for her/himself ' (1993, p. 182). In my research, 
I did interpret research material gleaned from students 
and research assistants and other lecturers and tutors, but 
only to elucidate my own autobiography of the study. 
Nor was it my objective to generate and test assertions 
such as "using this or that technique can lead to balanced 
attitudes". 

Comte: Was your research a case study? Did it involve 
ethnography? 

Mark: Because I concentrate on giving an 

autobiography of my own personal history, my study is 
not an ethnographic study. In other words, the primary 
purpose is not to give "analytic descriptions or 
reconstructions of cultural scenes and groups" (Goetze 
& LeCompte, 1984, p. 2). Indeed, my approach does not 
fit neatly into Goetze and LeCompte's classifications 
(1984, Chap. 1). Neither was my research a case study 
according to Lancy's definition (1993, p. 183). 

Comte: Quite unusual. So you did not adhere 

consistently to your original research hypothesis? 

Mark: I took it that in cohducting qualitative 
constructivistresearch, it was legitimate to allow for the 
evolution of research interests and foci (Lincoln & 
Denzin, 1994; Erickson, 1986;Guba & Lincoln, 1989). 
In my qualitative research from 1991 to 1993, my 
original intention was to conduct pre-intervention and 
post-intervention surveys supported with qualitative 
structured interviews to investigate my early implicit 
hypothe sis that discourse could balance technicism in the 
teaching-learning process of University Information 
Systems tutorials. In the early stages, I was particularly 
interested in "discovering and communicating the 
meaning-perspective of the people studied" (Erickson, 
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1986, p. 123). In this I included myself. Overall, my aim 
was "not proof, in a causal sense, but the demonstration 
of plausibility" (Erickson, 1986, p. 149). But that first 
phase of the research was so flawed that I moved on to 
the second phase of psychologically-oriented heuristic 
reflection (Moustakas, 1990). I have a hope that I have 
been part of what Daft and Lewin (1990) comment as: 
"the significant discoveries, the best science, require us 
to be more venturesome and heretic in research design, 
and to explore fundamental questions without knowing 
the answer in advance" (p. 7). 

Comte: Your first phase sounds very much like Thomas 
Barone (1987a) research whichRist (1 987) condemns as 
"all the negative critiques of qualitative work come 
home to roost in the article it is impressionistic, lacks 
any evidence of considering either the reliability or the 
validity of the data, the methods of analysis appear 
entirely idiosyncratic to the author, and the research 
question seemed framed once the study was over" (p. 
448). 

Mark: Yes, I have sympathy with Risfs (1987) critique 
of Barone's (1987a) research as being "hit and run" 
fieldwork (p. 448). (That is not to say that Barone's 
(1987b) reply to Risk does notanswer the charges.) The 
difference is that my two years of the first phase of 
qualitative research was no "blitzkrieg ethnography". 
However, it was some of the other concerns raised by 
Rist that reinforced my decision to abandon the first 
stage of the qualitative research and conduct the 
psychologically-oriented heuristic reflection (Moustakas, 
1990). This reflection led me to make three major 
heuristic impressions. 

Comte: All very interesting Mark, but how did you 
justify these impressions. Did you triangulate by using 
multiple data collection methods and sources and by 
usingmultiple methodologies and analyticalapproaches 
to ensure that you obtained the truth about what 
happened in your classes and your teaching and your 
inner awareness? 

Mark: I can see thatjustification, truth and a fixed point 
of reference to enable triangulation is important foryou, 
Auguste. I prefer to use Richardson's (1994) concept of 
crystalization rather than the notion of triangulation. 
Like a crystal, my research is a postmodernist mixed- 
genre text which reflects and refracts ever-changing 
pictures and images of my central themes. I did not 
rigorously triangulate to prove objective truths about 
technic ism and open discourse. 



However, as Schaller andTobin's (in press) recommend, 

I referred to the "quality criteria for interpretive 
research", which were gleaned mainly from Guba and 
Lincoln's (1989) work. "Through careful attention to 
fairness and the ontological, educative, tactile and 
catalytic criteria for establishing the goodness and, 
hence, the authenticity of the interpretive research", I 
endeavoured to adhere (albeit in an unusual way, and 
with some failures) to the six procedural steps of 
prolonged engagement, persistent observation, peer 
debriefing, negative case analysis, progressive 
subjectivity, and member checks. I did endeavour to 
show that my impressions and suggestions were 
sufficiently viable to be taken seriously, with sufficient 
and clearly specified warrants. However, I concluded 
that I could not claim the degree of authenticity or 
trustworthiness which Schaller and Tobin suggested is 
necessary for interpretive research. 

Comte: Even more interesting Mark. Why then, if you 
could not claim trustworthiness, did you describe these 
'impressions' in your thesis? 

Mark: I think that it is a question of the degree of 
authenticity that some authorities expect to see 
demonstrated in research. I certainly could not claim the 
degree of trustworthiness that you seem to expect, 
Auguste. I did demonstrate some degree of adherence to 
the ontological, educative, tactile and catalytic criteria 
that Guba and Lincoln (1989) contend is sufficient for 
authenticity. However, what I think is more im portant, is 
an understanding of the nature of heuristic inquiry and 
heuristic reflection. Inc orporating art, poetry and science 
(Morse, 1 994, p. 225), I focused on gleaning impressions 
about technicism and discourse. While researching, I 
found myself resonating with the key phrases that Tesch 
takes from Moustakas (1981, p. 212) to describe 
heuristic inquiry "an integrative living form", "being 
involved, committed, interested, concerned", "intuitive 
visions, feelings, sensings that [go] beyond anything 
[one] could record or think about or know in a factual 
sense"; from Colaizzi (1978, p. 67) "imaginative 
presence", "wondering about"; and from Douglas and 
Moustakas (1985) "examining with a sense of 
wonder". With discovery being the emphasis rather than 
verification, the essence lies in intuition as inner 
contemplation "at the confluence point of the social 
sciences and the humanities" (p. 69). My method of 
reflection, based around insights from Jungian analytical 
psychology, is similar to what Colaizzi (1978) terms 
"phenomenological reflection". As Tesch (1990) 
describes, in phenomenological reflection: 
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The phenomenon rises to clearer awareness by 
approaching it from different angles, for 
instance from a metaphorical, or a mythical, or 
a poetic perspective. Phenomenological reflec- 
tion is a process of "wondering about" and 
searching, delving into a phenomenon, awaking 
to it, and letting oneself be inspired (p. 70). 

Reflective phenomenology "aims at a descriptive 
understanding of psychological phenomena by 
reflectively disclosingtheir meaning" (Colaizzi, 1 978, p. 
68). I do not concentrate on a descriptive understanding 
of the experience of inner technicism. I concentrate on 
giving descriptive impressions about technicism and 
open discourse, based on my heuristic inquiry and 
psychological self-study which included inner discourse. 
I do not concentrate on giving impressions about my 
experience of technicism or open discourse or inner 
discourse. I understand this as the major difference 
between my heuristic inquiiy (Moustakas, 1990) as 
distinct from reflectivephenomenology (Colaizzi, 1978; 
Moustakas, 1994). Moustakas (1956, 1959, 1966, 1969 
a & b, 1981 a & b, 1990, 1994) has been a pioneer in 
developing and applying both heuristic inquiiy and 
reflective phenomenology. 

Tesch (1990, p. 171) commented that researchers 
conducting reflective phen omen ology do not necess arily 
adhere to some of the canons of trustworthiness that 
have been established for qualitative research. She adds 
that, when considering the results of the research, it is up 
to the reader: "to decide for yourself whether you want 
to consider them to be 'legitimate’" (p. 171). Her 
comment holds true for my heuristically gained 
impressions, and that is partially why I was bold enough 
to describe them. 

Comte: So you are saying that your appropriation of 
somekind of mix of heuristic inquiry, and psychological 
reflection, enabled you to find justification for 
describing your impressions. I hope that there is more 
reason than this. 

Mark: Yes Auguste, there is something more. The 
substantive issue, at the heart of m y research , evolved to 
be an investigation of technicism and open discourse 
from both an inner, psychological perspective and an 
outer, educational perspective. The inner strategic 
rationality and reflective inner discourse related to the 
outer technicism and open discourse in the University 
Information Systems education. I dialogically move 
between describing impressions of technicism and open 



discourse in the computing classes and the inner 
technicism and inner discourse of my pedagogical self. 
As we all do, to a greater or lesser degree of 
consciousness, I carried out a dialogue between that part 
of me experiencing the outer world and my inner world. 
The difference was that I consciously used this dialogue 
to gain descriptive impressions about the nature of 
technicism and open discourse. 

Comte: How exactly does one gain an heuristic 
impression? 

Mark: Moustakas (1990) states that there are usually 
four phases to an heuristic inquiry: (1) framing the 
questions; (2) carrying the questions within; (3) listening 
to other voices; (4) tying it all together. The framing of 
my questions took place in late 1993 to 1994 as I 
reflected on ethical flaws in my 1991 to 1993 qualitative 
study. By carrying thequestion within,! experiencedall 
the aspects of this phase as described by Moustakas 
(1990, pp. 28-31) as immersion, incubation, 
illumination, insights, and explication. In the insights 
process, I dialogically gained insights into both theinner 
and outer nature of technicism and discourse. Aspart of 
the explication process, and the ’tying it all together’ 
phase, I identified and then explained the insights that 
seemed to me to be most relevant to responding to the 
research questions. Moustakas explains that the tone of 
the heuristic research tradition is to offer responses, 
insights or conclusions to the research questions. The 
phrase "impressions about technicism and open 
discourse" seemed to me to be more elegant than 
"insights into the nature of technicism and open 
discourse". After some thought, I came to the conclusion 
that the word impression was a justifiable synonym for 
the wo rds M oustak as uses . 

Comte: But did the research make any difference to 
yourself, or to anybody, with respect to information 
systems? 

Mark: For myself, I found the main change to my 
information systemsteaching came from being inspired 
to become what Schon (1983) terms a reflective 
practitioner. I was also inspired by Mark von Wodke's 
(1993) book. Mind over Media: Creative Thinking Skills 
for Electronic Media. I introduced the physical 
stretching and mental relaxation exercises suggested by 
von Wodke (1993) into the computer laboratories. Von 
Wodke asserts that ergonomic, relaxation and 
imagination activities, even to the point of including 
Jungian dream interpretation and journal writing. 
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encourage creativity and mental and physical health 
whilst using computers. For example, as the computers 
were turning on, I encouraged the students to sit straight, 
close their eyes and breath slowly and deeply while 
silently repeating "calm body, alert mind" while 
breathing in and letting go of tension while breathing 
out. As the application was loading up from the menu I 
asked them to join with me in stretching across to the left 
and then to the right. I stressed the value of ergonomic 
approaches and the resting of eyes and body . In my post- 
graduate seminars, and even at times in the 
undergraduate mass lectures, I guided the students in 
brief mental relaxations. At times I included 
visualisations such as imagining a splendid flower 
blossoming on the computer screen or imagining a rural 
walk with sun and grass and flowers and breeze. 
Professor Robert Flood (verbal answer to a question at 
the keynote address of the Australian Systems 
Conference, 26 September, 1995) refers to these sorts of 
practices as physical and mental ergonomics. T 
encouraged the students to read von Wodke's (1993) 
book, and even suggested they try out his ideas on 
reflective journals and dream journals. 

Comte: Well, I guess it takes all types to make up the 
world. But do you have any conclusion from your study? 

Mark: I have already stated my three impressions. My 
third impression, about my innertechnicism, isabout the 
psychology of doing research. My tentative conclusion 
is that the deeper levels of the human psyche, if not 
recognised and honoured in self-reflective practice 
(Schon, 1983), will (paraphrasing Norris, 1992, p.l80) 
disrupt even (especially?) our most noble and well- 
framed attempts at educational reform. As a further 
research interest, T am exploring the idea that 
unconscious agendas play a role in the success or failure 
of information systems projects (Campbell Williams, in 
press b). 

SUMMARY 

In this paper, I mention some aspects of the heuristic 
inquiry which I undertook as a consequence of my 
heuristic analysis of my flawed qualitative project of 
1991 to 1993. I was led to dialogically explore 
understandings and experiences of technicism and open 
discourse in the outer world of the University 
Information Systems teaching reform, and in the inner 
world of my own pedagogical self. This is a complex 
notion. 



Jung (1968) contends thatthe dialogue between the outer 
and inner worlds, the conscious and the unconscious, 
although of vital importance in almost every area of 
human pursuit, is often neglected, usually at a cost. I 
explored this dialogue in the context of reflecting 
heuristically on a teaching reform in a University 
Information Systems course. I think that the conscious- 
unconscious dialogue is present in any research, and is 
pervasive in academ ia, but largely ignored, usually at a 
cost (Campbell Williams, in press a). For some 
researchers the cost may be high. What price is the 
self? or the soul (Campbell Williams, in pressb)? For 
science in general, some would contend that lack of 
reflectivity, including lack of awareness of the 
unconscious, lies near the heart of som e failures, perhaps 
the failure, of the scientific project of modernity (Adorno 
& Horkheimer, 1992; Bowers, 1992; Habermas, 1972; 
Marcuse, 1964, 1969). 

Heuristic research, even in management information 
systems research, is not: 

"Madness, you mean?" said the Professor quite 
cooly. "Oh, you can make your minds easy 
about that. One only has to look at her and talk 
to her to see that she is not mad." 

"But then," said Susan, and stopped. She had 
never dreamed that a grown-up would talk like 
the Professor and didn't know what to think. 

"Logic!" said the Professor half to himself. 
"Why don'tthey teachlogic atthese schools?..." 

"But do you really mean, sir, " said Peter, "that 
there could be other worlds all over the place, 
just around the corner like that?" 

"Nothing is more probable," said the Professor, 
taking off his spectacles and beginningto polish 
them, while he muttered to himself, "I wonder 
what they do teach them at these schools" 
(Lewis, 1950, p. 49). 

In this paper, I have endeavoured to demonstrate that, 
indeed, there are other worlds of research "all over the 
place, just around the corner" (Lewis, 1950). The 
research world of heuristic inquiry probably seems 
strange to some, as, I guess, does my research writing 
style. But there remains a serious question: "Can 
heuristic research and unusual research writing lead us 



26 



Proceedings of the 15^’ Annual Conference of the International Academy for Information Management 



o 

ERIC 



36 



to new truths? Or, perhaps, to revisit old truths, in new 

ways?" 
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ABSTRACT 

A promising recent development in tertiary education involves the application of problem-based learning as a 
curricularvehicle to develop student talen t. Problem-based learning (PBL) is common in professional education, such 
as in medical, law, and business schools, and is becoming increasingly common in pre-college education. However, 
it is less comm on in information systems education. We have successfully applied PBL to information systems courses, 
and are reporting our results in this paper. Moreo ver, there is a m ajor difficulty in applying PBL as there is no formal 
methodology of assessing.students ' work. This paper explores this issue by creating an instrument for assessm ent in 
a PBL setting. The instrument has been specifically designed to evaluate the generic abilities and skills ofinformation 
systems graduates. We also recommend that the instrument or a modified version of the instrument be used for 
assessing other subject areas when the PBL is applied. The instrument has been proved to be a successful tool for 
assessment of the PBL. The result of applying this assessment instrument to our information systems courses will also 
be pres ented. 



INTRODUCTION 

Problem-based learning (PBL) isan educational strategy 
that uses problems as the starting point for student 
learning (Bligh, 1995). It is a curriculum design and 
teaching/leaming strategy, which recognizes the need to 
develop problem-solving skills as well as the necessity 
of helping students to acquire the necessary knowledge 
and skills (Boud & Feletti, 1997; Biggs 1999). The m ain 
issue is to reduce direct instruction as students assume 
greater responsibility for their own learning. Students are 
given ill-structured problems through which they 
develop high-orderthinking and problem-solving skills. 
The shift in the teaching and learning process is more 
student-centered than teacher-centered. The role of the 
teacher is to encourage student participation, provide 
guidance to students, offer timely feedback, andassume 
the role of learner as well (Aspy et al., 1993). Evidence 



of the success of problem-based learning as an 
instructional strategy is strongly positive, particularly in 
fostering increased knowledge retention (Norman & 
Schmidt, 1992), encouraging general problem-solving 
skills, for deep-biased students (Norman & Schmidt, 
1992; Lai, Tiwari & Tse, 1 997), promoting self-directed 
learning skills, and increasing intrinsic interest in the 
subject matter. Since its introduction as an instructional 
method used in the medical school at McMaster 
University in Ontario in the 1960s, PBL has spread to 
numerous educational institutions around the world. To 
the best of our knowledge, the PBL strategy has not 
become so popular in the field of computer science and 
information systems. We have appliedthe methodology 
for the first time in our respective institutions quite 
successfully. The course that we applied it in our 
discipline was Systems Analysis. Students had to take 
greater responsibility for coming up with a solution for 
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a given problem case. The details of this application 
follow this section. 

Aligning PBL activities and subsequent student 
assessment often proves to be difficult for teachers, with 
many PBL activities followed by traditional assessment 
confusing students by dismpting their understanding of 
teacher expectations. Traditional assessment generally 
requires that students memorize the content of teaching 
materials without understanding. Once the examination 
is over, everything has been forgotten. Students learn to 
pass examinations without understanding what they 
should have learnt. On the other hand, if assessment 
requires students to solve problems, then they collect as 
many sample solutions ofproblems as they can. The key 
issue is to assess students so that they learn what we 
want them to learn. Quite a few PB L assessm ent models 
have been adopted, but they may not be appropriate to 
assess the generic quality of IS graduates. In the 
remainder of this paper, we describe some of the most 
recentPBL assessment strategies that have been adopted 
by different scholars, present an instrument which is 
particularly designed for the assessment of IS subjects 
when the PBL approach to teaching is applied, and 
finally our experience of applying this assessment 
instrument to IS courses in the fall of 1 999. The main 
focus of this paper was the development and 
implementation of an assessment instrument when the 
PBL strategy is applied. 

PBL ASSESSMENT MODELS 

The study of problem-based learning assessment has 
been taken in two different directions by researchers; 
namely assessing the value of a problem-based learning 
curriculum, and student assessment in a problem-based 
learning approach. In this work we are concerned with 
the latter issue that is how to assess students’ work. 
When we first applied PBL to our information systems 
courses, the first problem that we encountered was how 
to assess students’ work in a PBL setting. Many 
instructors and researchers who apply or study the 
assessment ofPBL have concerns about these issues. For 
example. Woods (1996) asks, “how do we handle tests 
in the context of small group, self-directed, self- 
assessment PBL?” There is, however, no general and 
systematic answer to this question, because students may 
be working alone, or in teams, and doing work that 
doesn’t fit into readily assessable products. Researchers 
have been working on PBL assessment for many years 
and have taken different approaches. These diverse 



approaches in PBL assessment as pointed out by 
Sundberg (1999) are due to different factors that will 
impact on the success of student-active innovation. 
These include the culture of the institution, the 
department, or even a particular subject. Reis and 
Renzilli (1991) claim that the major weakness of 
historical and contemporary PBL efforts is the lack of 
formal student evaluation. In the literature, there are 
many studies of PBL assessment in different subject 
areas. Generally speaking, they all share the same 
concern. However, the approaches responding to these 
concerns differ from subject to subject and from 
institution to institution. In the following we present 
some of the most recent research results. 

Nowak and Plucker (1999) provide suggestions for 
aligning instructional activities and assessment in the 
PBL process. This suggestion assumes three categories 
of students. In the first option, the assumption is that 
students are professional in the subject area of the 
instruction, and the instructor is their supervisor. The 
second suggestion is that the PBL assessment should be 
structured in the area of the student outcome. They 
provide reasonable guidelines regarding the instructor’s 
expectation from the students. Finally, the third 
suggestion states that instructors should hold off an 
assessment until the end of the activity or until the unit 
of work is complete. 

In the mental measu rements yearbook, Buros (1 999) has 
compiled several instruments that can be used for PBL 
assessment. Many faculty of McM aster University have 
adopted a modified version of these instruments. Some 
of these instmments include testing of subject 
knowledge, problem-solvingskills,metacognitive skills, 
lifetime learning skills, and critical thinking skills. 

Hicks (1998) classifies assessment of a PBL subject into 
four categories, namely teacher observation, student- 
produced written material and products, peer and self- 
evaluation, and feedback from the outside community. 
All of these four different classifications have the same 
outcome objectives, such as basic skills, and critical 
thinking skills. Hicks points out thatthe hardest to assess 
are the attributes and disposition of students, such as 
empowerment, diligence, empathy and pride. 

Woods (1998) has proposed several units of skills for 
assessment of chemical engineering coursesatMcM aster 
University. These units focus on individuals solving 
relatively well-defined problems, interpersonalskills and 
group problem-solving, and messy problem-solving 
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skills. Woods (1996) has also provided a general 
guideline for PBL assessment directed mostly towards 
chemical engineering courses. 

The SOLO taxonomy (Boulton-Lewis, 1998) is based on 
the study of outcomes in a variety of academic content 
areas. It stands for Structure of the Observed Learning 
Outcome. It has five levels of taxonomy: p restructural, 
unistructural, multistructual, relational and extended 
abstract. It is intended to measure the students' 
quantitative and qualitative increase in understanding of 
the subject. In quantitative terms, it measures how much 
students can memorize and recognize terminology and a 
disorganized set of items. It also measures the students' 
performance in qualitative terms. That is how students 
relate, apply, generate, reflectand theorize of the subject 
taught. At this level, students are able to apply theory to 
practice for problem-solving. 

Assessment by portfolio (Gibbs, 1998) requires the 
student to be acquainted with the course objectives and 
is asked to provide evidence that learning relevant to 
those objectives has been achieved. This requires the 
learner to recognize the nature and quality of his or her 
own learning. The student presents his or her best 
‘learning' against the objectives. Students have to use 
their judgments in assessing their own work. There are 
two drawbacks. First, the students may claim something, 
which he or she did not do. Second, the portfolio may 
require that students work excessively and create work 
both for themselves and for the teachers. 

Triple jump (Feletti, 1997) uses a three-step exercise, 
with the student evaluated at each step: step one deals 
with the problem case; diagnosing, hypothesizing, 
checking with the database, use of information, 
reformulating. Step two tests them on independent study: 
knowledge gained, level of understan ding, evaluation of 
information gained. Step three is concerned with final 
problem formulation: synthesis of key concepts, 

application to problem, self-monitoring, response to 
feedback. 

Researchers at the University of Wisconsin (Stone, 

1 996) have adopted a new PBL assessment that they call 
ability-based assessment, in which they have identified 
nine generic abilities that amedical graduate should pass 
to be admitted into residency training. These abilities 
include appraisal, analysis, assessing own and peer 
performance, self-directed learning, handling stress, 
completing tasks, communications, consideration of 
professional ethics in decision-making, and interpersonal 



skills. Multiple-choice testing has been replaced by this 
activity-based assessment. 

(Trevitt C. and Pettigrove M., 1995) have been using a 
combination of assessm ent and self-assessm ent to a fire 
science and management course using five criteria, 
namely class tests, oral debrief exercises, district 
committee fire management plan proposal, take home 
assignments and presentation. The main goal of their 
work is to increase the emphasis on inducing students 
into the process of self-evaluation according to specified 
criteria. 

The above PBL assessment strategies have been 
designed for specific objectives and for specific 
disciplines. To the best of the authors' knowledge, no 
PBL assessment tool has been develope d for information 
systems courses. When we first applied PBL teaching 
strategy to our courses the assessm ent of students' work 
was a major concern for both students and instructors. 
Students were concerned how their work would be 
graded and instructors were also concerned bee ause there 
were no formal test or quizzes. To make the application 
of PBL strategy to our courses more practical and 
successful we have developed an assessment instmment 
for information systems courses that we have applied to 
our courses. The details of how the instrument was 
developed and subsequently applied are given below. 

PBL ASSESSMENT INSTRUMENT 

Information systems graduates (specially systems 
analysts) must have certain skills and ability and be 
prepared for industry. There is little doubt that 
information systems graduates should be able to 

• work independently and in a group environment, 

• solve a problem logically and systematically, with 
little or no supervision, 

• understand how to acquire knowledge forproblem- 
solving, 

• have critical thinking and problem-solving ability, 
and 

• communicates effectively (both orally and in 
writing) 

Students mustbe equipped with the academic knowledge 
and the practical skills forproblem-solving. Theymust 
understand how the theory is applied to practice, and 
know the circumstances necessary for achieving the cost- 
effective solution. There are several issues that concern 
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a teacher: first, what the students need to know 
academically; second, the skills they need to have; third, 
how they apply their skills and academic knowledge to 
know effectively when and why it is appropriate to do 
these things? The third should be an alignment between 
the curriculum objectives andassessm ent. That is to say, 
how well students know thesubject, how well they have 
learnt, and at what level. 

PBL activities support the development of ‘preferred IS 
graduates’, which also leads to long-life learners. When 
the PBL approach is adopted, there is some concern 
about how students are to be assessed with a practical 
and manageable assessmentmechanism.The assessment 
of students should be aligned with their generic abilities. 
The assessment techniques mentioned above have 
targeted specific subject areas that are not necessarily 
suitable for IS courses. Thus, based on the previous work 
done by other researchers, we have created an 
assessment instrument specifically for IS courses. The 
instrument consists of a list of nine criterion skills, listed 
in Table 1 . For each criterion skill, we have proposed a 
set of tasks that must be fulfilled to develop the given 
criterion skill. These tasks are listed immediately 
following Table 1. Aninstructor can use some or all of 
these tasks to assess students’ work for that skill. The 
authors used this instrument in theirinformation systems 
courses in the fall of 1999. The outcome of this 
experiment follows the list of criterion skills. Although 
the focus was on IS courses, most of these criteriacan be 
used for other subject areas as well. 

Table 1 shows the criteria that have been developed and 
used by the authors to assess students’ work in 
information systems courses. For eachcriterion skill, we 
have developed a set of tasks that can be used for 



assessment. The details of these tasks are listed in 
Appendix A. 

APPLICATION OF PBL AND 

ASSESSMENT INSTRUMENT 

The authors have taught system s analysis and design for 
a number of years using the traditional approach. This 
subjectis taught atthe upper level of a computer science 
(CS) and IS program. In an effort to develop students’ 
talents and enhance their learning, the authors decidedto 
apply the PBL strategy to the teaching of this course in 
their respective ins ti tut ions for the first time in the fall of 
1999. The application of PBL was done with close 
coordination and communication between the authors. 

The systems analysis course in which we chose to apply 
PBL is offered once ayear. The course meets three times 
a week, two of which are lectures an d the third meeting 
is lab/tutorial. The course was designed to be project- 
oriented, that is there was no formal test or quizzes, but 
all course requirements were fulfilled through projects. 
Every week there were two hours of lecture and one hour 
of lab/tutorial. Students worked in groups of up to six. 
They were assigned several ill-structured problem cases 
during the semester. For each problem case, students 
worked together to come up with a solution. During the 
lab/tutorial hour, the students worked on a certain aspect 
of their project that required group meeting, such as 
dividing the work among them selves, and setting future 
meeting dates. The instructor took the role of a coach 
monitoring their work. 

Students were assigned several ill-structured problem 
cases. For each problem case, they were required to 
submit both individual and group work in the form of 



TABLE 1 

LIST OF CRITERION SKILLS FOR PBL ASSESSMENT 



No 


Criterion S kills 


Description 


1 


Problem understanding 


Evidence of problem understanding and independent study 


2 


Use of resources 


Evidence of determination & utilization of information resources 


3 


Teamwork 


Evaluation of student’s teamwork success 


4 


Critical thinking 


Evidence of practical and optimal solution 


5 


Management skills 


Setting deadlines; finishing on time 


6 


Writing sk ills 


How well and professionally the work iswritten 


7 


Oral presentation 


How presentation is done 


8 


Team communication 


How well the student communicates with team members 


9 


Self-assessment 


Seriousness and accuracy of self-assessment 
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log their projects solutions as well as their projects. The 
assessment instrument have been used to grade students’ 
work. Each question in our assessment instmment were 
given a weight from 5 to 1 with 5 being the highest score 
and 1 being the lowest. This grading scale was used to 
grade their project work. 

In the early stage of the PEL approach introduction, the 
change of students’ attitude towards the subject and 
especially towards the learning process became obvious. 
Students found the learning activities, such as teamwork , 
critical thinking, and problem-solving more meaningful 
than the traditional approach of lecturing and taking 
multiple choice or short answer tests. Nevertheless, the 
students’ main concern was assessment. Questions like, 
“How will you grade our w ork?” were raised frequently. 
Application of assessment instrument reduced student’s 
anxiety significantly. This task could on ly be taken care 
of so quickly through the coordination and 
communication of the authors. After their first project, 
they developed more faith in our assessment 
methodology and were convinced that the assessment 
was relatively fair and accurate. For their subsequent 
projects, they showed less anxiety about their grade and 
focused on their learning activities. Students spent more 
time studying and preparing their projects in the PEL 
approach than in the traditional approach. The major 
focus of this course was the development and application 
of assessment instruments in PEL. The learning outcome 
has been very positive, and the informal feedback of 
students was favorable. Here are some quotes from 
students in our courses “ I learned systems analysis 
topics without much memorizing’', “ I enjoyed working 
in a team”, “PEL requires hard work and is a new 
learning experience”, and “The course was challenging 
and fun”. It is planned to have a formal survey on the 
students’ perception of our adopted assessment 
instruction, the result of which will be presented in 
another paper. 

CONCLUSION 

We have successfully applied the PEL strategy to IS 
courses and it was an exciting experience for both 
students and instructors. It proved to be superior to the 
traditional approach to teaching in many ways. First, 
students’ learning activities, including reading, 
searching, and writing, were much higher than when 
traditional teaching was applied to the same course. 
Second, the level of interaction among students was 
much higher. Third, the students had to keep a portfolio 
of their learning activities, which assisted them to be in 



control of the topics to be learnt. The major concern 
among educators who apply the PEL strategy is 
assessment of students’ work. Evaluation of student 
achievement is an important aspect of education, and the 
skills required for solving real-world problemsmust be 
included in that assessment. In addition, alignment of 
instruction and assessment is essential when the PEL 
approach is to be used. In response to this important 
concern, we have developed and applied an assessment 
instrument that is targeted towards the generic skills of 
IS graduates. The assessment instrument has created a 
unified framew ork for the in structors to ev aluate 
students’ work; otherwise the assessment would have 
been very difficult. It has proved to be successful in 
many ways. It has significantly reduced students’ anxiety 
with regard to the accuracy and methodology of 
assessment as they realize the need to work towards the 
generic abilities of IS professionals. The instructors had 
a better ground for evaluation with a checklist that is 
easy to use for those who wish to apply the PEL strategy 
in their courses. Although IS graduate skills are the 
target of our instruments, it can also be fully or partially 
used for assessment of other subject areas. We have 
provided a useful tool for instructors who plan to apply 
PEL in their courses 
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1 . Problem Understanding Objective : 

Students should be able to: 

• Describe all the aspects of the problem 

• Have a good idea of what needs to be done to solve the problem 

• Have created strategies to solve the problem 

• Have demonstrated the ability to work with limited supervision 

2. Use of Resources Oblective : 



Students should be able to: 

• Prepare a printed copy of the needed material for his/her reading 

• Demonstrate his/her attempt to create other resources 

• Provide an accurate summary of the search work 

• Demonstrate evidence of electronic search 

• Relate the prepared information to the given problem 

• Evaluate the accuracy and credibility of the searched information 

• Create a relation between the prepared information and the real world problem 

3. Teamwork Objective : 

Students should be able to: 

• Demonstrate effective communication 

• Contribute at least his/her share of the work 

• Cope with conflicts 

• Evaluate team members’ work fairly and honestly 

• Share information with team m embers 

• Demonstrate leadership ability 

• Initiate discussion 

• Help other team members when they are in difficulty 

4. Critical Thinking Objective : 

Students should be able to: 

• Propose solutions to the given problem 

• Test their proposed solution 

• Propose alternate solutions 

• Provide reasons or justify the solutions 

• Present some background that support the solution 

• Evaluate and commend his/her solutions and the work of others 

• Present ideas/comments that are useful and constructive 

5. Project Management Objectives : 

Students should: 

• Have a good plan, schedule, and means of controlling the project 

• Be able to arrange tasks logically and systematically 

• Estimate resources reasonab ly 

• Submit their work on time 

• Provide a log of all m eetings and com munications w ith other team m embers 
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• Provide documented evidence of good communication with team members and outside organizations 

• Present a record of their submitted work 

6. Writing Objective : 

Students should: 

• Produce a professional report 

• Provide a good introduction and summary to their report 

• Perform a good problem analysis 

• Produce a report with good content 

• Provide alternative solutions with justification 

• Draw a reasonable conclusion 

• Provide references in their report 

• Demonstrate the ability to convert information into a meaningful report 

• Demonstrate any potential prospect for being successful systems analysts 

7. Oral Presentation Objective : 

Students must: 

• Explain their work clearly 

• Present the essential points of their work clearly 

• Use effective visual aids to support the presentation 

• Be able to answer questions during presentation 

• Be able to behave professionally during presentation 

• Be able to paraphrase their understanding of the topic during presentation 

• Be able to transfer their knowledge to the students in an appropriate way 

8. Team Communication Objective : 

Students must: 

• Have a good understanding of other team members’ suggestions and points 

• Demonstrate evidence of useful and constructive discussion 

• Make suggestions that provide guidelines to the team 

• Support their points through discussion 

• Provide evidence of making constructive suggestions 

• Provide concise and short messages and yet be able to resolve problems 

9. Self-Assessment Objective : 

Students should provide: 

• A list of what he/she contributed 

• A list of important topics learnt 

• A list of accurate assessment of other team members 

• A list of what he/she did not learn 

• A list of the meetings he/she did not take part in 

• A list of items that other students have done on his/her behalf 
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This paper reports theresults of an empirical examination of the effects ofpersonal experience and social interaction 
on individual knowledge and performance in a specific decision making task context. The study revealed a differential 
effect of increased experience on the quality of participants’ decisions. In particular, increased experience did not 
result in significant decision improvement in the non-interactive social environment, but did so in the interactive 
social environm ent. The study also revealed that social interaction was beneficial irrespective of personal experience, 
and led to better performance at both low and high experience levels. The results suggest that individuals may ben efiit 
from, in combination, opportunities for personal experimentation and social interaction with others. These results 
may be useful to all those responsible for planning knowledge management strategies aimed at enhancing knowledge 
creation and utilisation . 



INTRODUCTION 

Information and knowledge management literature 
repeatedly report that despite large investments and 
proliferation of programs to improve individual and 
organisational performance, failed programs far 
outnumber successes and improvement rates remain low 
(Garvin 1 998; Sauer, 1993; Swanson, 1988). It has been 
claimed that itis becausethese companiesfailedto grasp 
the basic truth, that before people or organisations can 
improve they first must learn. The literature suggests that 
to remain competitive, or even to survive, in today’s 
uncertain economy with shifting markets, proliferating 
technology, multiple competitors and shorteningproduct 
life, companies will have to convert themselves into 
“learning organisations (Garvin, 1998), “knowledge- 
creating organisations (Nonaka, 1998), and 
“organisations of knowledgeable specialists (Drucker, 
1998). 



While there seems to exist a growing recognition of the 
importance of learning and knowledge for competitive 
advantage or survival (Devenport & Prusak, 1998; 
Davenport et al., 1998; Drucker, 1993; Grayson & Dell, 
1998; Stewart, 1997), the complexities of learning and 
the large number of interacting factors which affect 
individual and organisational learning present many 
challenges. Scholars define learning organisation as 
skilled at creating, acquiring andtransferring knowledge 
andmodifying itsbehaviouraccordingly (Garvin, 1 998); 
as a place where people continually expand their 
capacity to create desired results, new thinking patterns 
are nurtured, collective aspirations are set free and 
people learn how to learn together (Senge, 1990); and a 
place where inventing new knowledge is a way of 
behaving or being, and in which everyone is a 
knowledge worker (Nonaka, 1998). Some authors 
suggest that the typical organisation of tomorrow will 
resemble a symphony orchestra or the university in 
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which professional specialists will direct and discipline 
their own performance (Drucker, 1998). 

It is argued that any company that wants to compete on 
knowledge must learn techniques of knowledge creation. 
Some clues as to what knowledge-based organisations 
will require may come from other knowledge-based 
entities(e.g. university orsymphony orchestra). W estern 
theorists have been recently criticised for having too 
narrow view of organisation and knowledge (Nonaka, 

1998) . They tend to consider organisation as an 
information-processing machine and focus mostly on 
hard information stored in organisational codified 
repositories as a main source of “explicit objective and 
systematic knowledge. Japanese theorists, on the other 
hand, recognise the importance of “ta cit know ledge that 
people gain through experience and sharing (Nonaka & 
Takeuchi, 1995). Considering that between 40% 
(AAOTE, 1998) and 90% (Hewson, 1999) of the needed 
knowledge in organisations is tacit, it is not surprising 
that there is a growing interest among the practitioners 
and academics alike to better understand howto tap into 
the wealth of knowledge in people’s heads. 

It has been suggestedthat new knowledge always begins 
with the individual, and that making personal knowledge 
available to others is the central activity of the 
knowledge-creating company. The spiral knowledge 
model (Nonaka & Takeuchi, 1995) assumes that the 
process of sharing will result in the organisational 
amplification and exponential growth of working 
knowledge. Furthermore, it has been suggested that in 
order to build learning organisation the first step should 
be to foster an environment conducive to individual 
learning, that is allow experimentation and time to gain 
experience, and second, to open up boundaries and 
stimulate exchange of ideas (Garvin, 1998) . Yet, little is 
known of the ways in which tacit knowledge is actually 
acquired and shared, conditions under which this occurs, 
and the impact it has on performance (Baxter & Chua, 

1999) . 

It is important to recognise the value of knowledge and 
the need for better understanding of the potential of 
various knowledge-management initiatives to support 
knowledge creation. The main purpose of this study is 
help to provide an insight into two specific initiatives 
(opportunities for personal experience and social 
interaction) aimed at promoting individual knowledge in 
a decision-making context. Decision-making is regarded 
as a knowledge intensive activity. Decision-makers often 
gain knowle dge of and ability to implement strategieson 
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the task from experience gained through task repetition 
and from feedback. Their work also involves a 
significant amount of social interaction with their 
subordinates, peers and superiors. With the growing 
importance of fosteringlearning and sharing of personal 
knowledge in modern organisations it is of interest to 
examine two things in particular. Firstly, whether 
personal experience gained in iterative decision contexts 
may affect individual decision makers’ working 
knowledge and what impact this may have on the quality 
of their subsequent decisions. Secondly, what effects 
might social interaction with other knowledge workers 
have on the quality of these same decisions. 

PRIOR RESEARCH 

The following review presents and discusses relevant 
past theoretical and empirical literature on the impact of 
personal experience and social interaction on various 
aspects of individual decision-making, and develops 
specific research questions of interest for the present 
study. 

Personal Experience 

Learning theorists suggest that people should learn from 
experience gained through task repetition and from 
feedback to adjust their behaviour and improve 
performance over time. In decision making, some 
authors (Payne et al., 1988) claim that adaptivity may be 
crucial enough to individuals that they would guide 
themselves to it without the need for an' external prod. 
Other authors (Hogarth, 1981) believe that the 
availability of immediate feedback, in addition to the 
opportunity to take corrective action, is critical for 
effective learning. Adequate feedback is considered 
especially important for the correct assessment of the 
previous responses in situations where the subject is 
unfamiliar with the task or topic (O'Connor, 1989). 

Empirical studies indicate mixed findings. Some studies 
reported modest learning from experience in a series of 
judgement tasks in which participants were provided 
with multiple pieces ofinformationof varying accuracy 
at a cost (Connolly & Serre, 1984; Connolly & Gilani, 
1982; Connolly & Thorn, 1987). Behaviouralpatterns of 
real participants were consistent with computer 
simulation and resulted in improved effectiveness with 
experience. Although the improvement was real, a 
serious deviation from optimality remaine d. In contrast, 
a study conducted in a similar multivariate judgement 
task reported no learning at all (Connolly & Wholey, 
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1988), while the results of a judgemental adjustment 
study (Lim & O’Connor, 1996) indicated that 
information strategy even worsened and consequently 
performance declined over time. Many studies 
performed in the multiple cue probability learning 
(MCPL) paradigm have concluded that people can not 
learn when multiple cues are involved or non-linear 
relationships exit (for review see Brehmer, 1980). 

It has been suggested that feedback may enhance 
learningby providing an information about the task, task 
outcome, individual’s performance and/or decision 
process. From this information, through task repetition, 
an individual may learn to adapt, ie. maintain, modify or 
abandon strategy to improve task performance. In his 
reviewof a number of laboratory and field studies on the 
impact of feedback on task performance, Kopelman 
(1986) reported that objective feedback (defined as 
information about task behaviouror performance thatis 
factual and inconvertible), had a positive effect on 
performance. But it was stronger and more sizeable in 
the field than in laboratory. In choice task setting, Creyer 
et al. (1990) speculated that for an individual to adapt 
decision strategy to a particular decision task, he or she 
had to have at leastsome vague ideas aboutthe degree of 
effort and accuracy characterising his or her decision 
process. They examinedhow decision-makers learned to 
adapt when presented with explicit accuracy feedback 
and/or explicit effort feedback. The results indicate that 
feedback on accuracy led to more normative-like 
processing of information and improved performance. 
The role of accuracy feedback was greater when the 
decision problem was more difficult. Explicit effort 
feedback, on the other hand, had no impact on 
processingor performance regardlessof the difficulty of 
the problem. 

In contrast, findings from MCPL studies with feedback 
reported by Klayman (1988) indicate that people have 
difficulties learning from outcome feedback in these 
tasks, but they can learn more effectively when they are 
provided with cognitive feedback, such as summary 
analysis of how their past predictions differed from 
optimal. Feedback was found to have induced learning 
in the so-called cue discovery tasks. People were found 
to perceive the existence and direction of a cue-criterion 
relation, but have difficulties in learning its shape. A 
significantimprovement in predictive success over time 
was attributed to cue discovery rather than accurate 
weighting. 



In a recent study examining the impact of several types 
of feedback on the accuracy of forecasts of time series 
with structural instabilities, Remus et al. (1996) found 
that task information feedback (showing the underlying 
structure of the task) with or withoutcognitive feedback 
(prompting on desirable behaviour) gave significantly 
better forecasting performance than the baseline simple 
outcome feedback. Adding cognitive information 
feedback to task information feedback did not improve 
forecasting accuracy. The study also found that 
performance outcome feedback (prompting with 
graphical indicator or words expressing levels of 
forecasting accuracy) was not superior to outcome 
feedback. The results for task and cognitive feedback 
largely replicated those by Balzer et al. (1992, 1 994). 

Some studies have found a detrimental effect of 
feedback on performance. In a complex probability task 
a large error on a particular trial might imply poor 
strategy or merely the fact that occasional errors are due 
to be expected in the probability task. As a consequence, 
outcome feedback may sometimes have a detrimental 
effect on strategy selection. Peterson and Pitz (1986) 
discovered that outcome feedback increased the amount 
by which the decision maker’s estimates deviated from 
the model. Participants made predictions about a number 
of games won by a baseball team during the year on the 
basis of a number of performance indicators. These 
findings were further reinforced by Arkes et al. (1986) 
who found that the omission of feedback was effective 
in raising performance. Those without feedback relied 
more on helpful classification rule that enabled them to 
judge well in 70% of their decisions, than those with 
feedback who tried to outperform the rule and 
consequently performed worse. 

In summary, findings concerning the impact of 
experience on decision performance in multivariate 
judgement tasks are mixed and inconclusive. The 
findings suggest that the quality of performance may be 
conditional upon the type of feedback, task difficulty, 
time period, and whether participants are allowed to 
experiment. Some theorists have suggested that 
“learning histories that capture past experiences may 
have a positive effect in terms of building of a better 
understanding about what works and what does not 
(Kleiner & Roth, 199 8). 
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Social Interaction 

Decisions in social environments can be made 
individually or in groups. The literature suggests that 
people make the majority of important personal 
decisions individually, but after significant social 
interaction (Heath & Gonzales, 1 995). Individuals often 
seek to consult with others before deciding what Jobs to 
take, what cars to buy, or what changes to make in their 
personal life. Managerial decision makers also follow 
similar interactive procedure when making business 
decisions. They collect information and opinions from 
their subordinates, peers and superiors, but make final 
decisions alone. Because they make their final decisions 
individually decision makers can use or ignore the 
information they collect during social interaction. In 
contrast, group decision making requires groups to reach 
a consensual decision. 

The information exchange model of social process 
suggests that people interact primarily for the purpose of 
information collection. Situation-theoretic approach to 
interaction (Devlin, 1999) assumes that, for most 
conversations, the aim of each participant is to take new 
information about the fo cal object o r situation into his or 
her context. The contextual situations can be represented 
graphically by ovals on a conversation diagram. The 
overlapping portions represent the contexts that the 
participants share. The interaction can be viewed 
geometrically as a gradual pushing together of the 
participants’ contexts so that the overlapping portion 
becomes larger. Persuasive arguments perspective 
(Heath & Gonzales, 1995) assumes that an individual’s 
position on any given issue will be a function of a 
number and persuasiveness of available arguments. It 
assumes that individuals come up with a few of 
arguments of their own, but during interaction they 
collect novel arguments and may shift their initial 
opinions. Group decision making approach recognises 
the collaborative nature of the interaction act and 
suggests the potential synergy associated with 
collaborative activity (Marakas, 1999). However, much 
of earlier research into group interactions questions the 
relative virtues ofcollaborative over individual decision 
making due to groupthink phenomenon. According to 
Janis (1982) members of the cohesive long-term groups 
strive for unanimity and do not realistically appraise 
alternative courses of action. This results in unfavourable 
outcomes. 

It is argued that situations where individual decision 
makers interact in a social environment but make their 



own decisions should be free from groupthink-style 
outcomes. In such situations, interaction is assumed to 
allow individuals to more accurately assess their 
information and analysis, and improve individual 
decisionperformance. A small number ofprior empirical 
studies investigating the issue indicate inconclusive 
findings. Heath and Gonzalez (1995) cited some earlier 
studies where interaction improved accuracy in general 
knowledge questions (Yates, 1991) and predictions 
domain (Hastie, 1986). However, their own research 
showed that individual performance did not improve 
much after interaction. Two studies on sportpredictions 
showed that interaction did not increase decision 
accuracy or meta-knowledge (calibration and resolution), 
while one study found little responsiveness in risky shift 
dilemmas. Instead, interaction produced robust and 
consistent increases in people’s confidence in their 
decisions. The authors offered a rationale construction 
explanation fortheir findings. It suggests that interaction 
forces people to explain choices to others and that 
explanation generation leads to increased confidence. 

In contrast, Sniezek and Buckley (1 995) reported mixed 
effects of interaction on individual performance within 
the judge-advisor system. In a given choice task 
concerning business events judges provided final team 
choices and confidence assessments independently or 
after being advised by two advisors. In particular, 
dependent judges made decisions as advised, cued 
judges made own decisions after being advised, and 
independent judges made initial decisions before and 
final decisions after being advised. Results showed that 
the effect of advice was dependent upon advisor conflict. 
With no-conflict, advice was generally beneficial. When 
conflict existed, it had either adverse (cued and 
Dependent) or no effect at all (independent) on judge’s 
final accuracy. 

In summary, interaction sometimes had positive, 
sometimes negative and sometimes no systematic effect 
on individual performance. The equivocal effects of 
interaction may be po tentially attributed to differences in 
size of interacting groups, expertise and status of the 
participants, or the characteristics ofthe tasks involved. 
Some studies indicate that as the number of participants 
increases, the likelihood of discussing unshared 
information decreases (Stasser et al., 1 989). According 
to recent studies on collaborativeproblem solving, teams 
of two people (dyads) are more successful than larger 
groups (Panko & Kinney, 1992; Schwartz, 1995). It has 
also been suggested that non-experts who are less 
informed about a decision problem should be more 
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responsive to information collected and pooled through 
interaction (Heath & Gonzalez, 1995), 

Study Focus 

The focus of this study is to investigate the two variables 
of personal experience and social interaction. 
Specifically, the study will explore whether personal 
experience of the decision maker gained through task 
repetition with immediate feedback and from learning 
history affects individual decision accuracy in a 
judgemental decision making task. The study will also 
examine whether, and how, the opportunity for social 
interaction affects individual decision performance. 
Finally, it will test the potential interaction effects 
between personal experience and social interaction, 

RESEARCH METHODOLOGY 
Experimental Task 

The experimental task in the current study was a simple 
production planning activity in which participants made 
decisions regarding daily production offreshice-cream. 
The participants assumed the role of Production 
Manager for a fictitious dairy firm that sold ice-cream 
from its outletat Bondi Beach in Sydney, Australia, The 
fictitious company incurred equally costly losses if 
production were set too low (due to loss of market to the 
competition)ortoo high (by spoilage of unsold product). 
The participants' goal was to minimise thecosts incurred 
by incorrect production decisions. During the 
experiment, participants were asked at the end of each 
day to set production quotas for ice-cream to be sold the 
following day. Before comm encing the task, participants 
had an opportunity to make five trial decisions (for 
practice purposes only). 

To aid their decision making, participants were provided 
with task relevant variables including actual local 
demand for product and three contextual factors that 
emerged as important in determining demand levels: the 
ambient air temperature, the amount of sunshine and the 
number of visitors/tourists at the beach. All contextual 
factors were dee med rela tively sim ilarly imp ortant. The 
task provided challenge because it did not stipulate 
exactly how information should be translated into 
specific Judgem ent. The pa rticipants w ere provid ed with 
the meaningful task context, sequential historic 
information of task relevant variables to provide some 
cues to causal relationship, and forecast values of 



contextual variables to suggest future behaviour. 
However, they were not given any explicit analysis of 
the quality of their information, or rules they could apply 
to integrate the available factual information. 

Instead, all participants had an opportunity to learn from 
own experience through task repetition and from 
performance history. Each participant was required to 
make thirty experimental production decisions over a 
period of thirty consecutive simulated days. While 
handling the task, different participants had different 
opportunity for social interaction with others. One half 
of the participants was required to make decisions 
independently, without any interaction with others. The 
otherhalf was encouraged to share theirinformation and 
opinions. More specifically, participants from this group 
were placed in groups of two and instructed to discuss 
their information and opinions before making their final 
decisions. However, they were not required to reach a 
consensual decision. 

At the beginning of the experiment, task descriptions 
were provided to inform participants about the task 
scenario and requirements. The given text differed with 
respect to the form of communication allowed. In 
addition, throughout the experiment instructions and 
immediate performance feedback as well as the history 
of past errors were provided to each participant to 
analyse earlier performance and to adjust future 
strategies. 

Experimental Design and Variables 

A laboratory experiment with random assignment to 
treatment groups was used, since it allowed greater 
experimental control. This made it possible to draw 
stronger inferences about causal relationships between 
variables due to high controllability. Independent 
variables were (i) personal experience (earlier phase vs 
later phase) and (ii) social interaction (constrained 
interaction vs encouraged interaction). 

In order to explore learning from personal experience, 
experimental trials were divided into two blocks of trials 
referred to as phases. Each phase was equivalent to a 
block of fifteen trials. Block 1 (or earlier phase) 
consisted of subjects’ first 15 trials, while block 2 (or 
later phase) consisted of their last 15 trials. Social 
interaction was m anipulated by com pletely constraining 
or maximally encouraging (through dialogue) sharing of 
ideas an d inform ation du ring de cision m aking. 
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Decision performance was evaluated in terms of decision 
accuracy operationalised by absolute error (AE) and 
symmetric absolute percentage error (SAFE). Absolute 
error was calculated as an absolute difference between 
the units of sales produced and the units of the product 
actually demanded (in hundreds of sale units). 
Symmetric absolute percentage error was obtained by 
further dividing the absolute error by an average of 
produced and demanded values and multiplying by 
100%. These measures have been suggested by 
forecasting literature (Makridakis & Wheelwright, 1989; 
Makridakis, 1993). Percentage error is generally 
preferred to absolute error because it controls for scale. 

Participants and Procedure 

The participants were 28 graduate students enrolled in 
the Master of Commerce course at The University of 
New South Wales, Sydney. Students participated in the 
experiment on a voluntary basis and received no 
monetary incentives for their performance. Generally, 



graduate students are considered to be appropriate 
participants for this type of research (Ashton & Kramer, 
1980; Remus, 1996; Whitecotton, 1996). The experiment 
was conducted in a microcomputer laboratory. On 
arrival, participants were assigned randomly to one of 
the treatment groups by picking up a diskette with an 
appropriate version of the research instrument. The 
instrument was specifically developed by one of the 
authors in Visual B asic. Studen ts were briefed about the 
purpose of the study, read case descriptions and 
performed the task. The session lasted about one hour. 

RESULTS 

The impact of personal experience and social interaction 
on individual performance was analysed statistically 
using the T-testmethod. A series of tests were performed 
to examine simple main effects ofindependent factors on 
two performance measures (absolute error and 
percentage error). Results are presented in Tables 1 and 
2 . 



TABLE 1 

PERSONAL EXPERIENCE EFFECTS 



Social Interaction 
Personal Experience 


Constrained 
Earlier Later 

(N=210) (N=210) 






Encouraged 
Earlier Later 

(N=210) (N=210) 






Dependent Variable 


Mean (std) 


Mean (std) 


t 


P 


Mean (std) 


Mean (std) 


t 


P 


Absolute Error 
(‘00) 


4.06 (4.20) 


3.62(4.01) 


1.10 


.273 


3.38 (3.23) 


2.33 (2.00) 


4.01 


.000** 


Percentage Error 
(%) 


19.25(25.16) 


16.36(21.59) 


1.27 


.207 


13.56 (1 1.25) 


9.28 (8.24) 


4.45 


.000** 


* p < 1.00 ** p < . 


05 




















TABLE 2 

SOCIAL INTERACTION EFFECTS 








Personal Experience 
Social Interaction 


Earlier 

Constrained Encouraged 

(N=210) (N=210) 






Later 

Constrained Encouraged 

(N=210) (N=210) 






Dependent Variable 


Mean (std) 


Mean (std) 


t 


P 


Mean (std) 


Mean (std) 


t 


P 


Absolute Error 
(’00) 


4.06 (4.20) 


3.38 (3.23) 


1.87 


.062* 


3.62 (4.01) 


2.33 (2.00) 


4.20 


.000** 


Percentage Error 
(%) 


19.25(25.16) 


13.56 (1 1.25) 


2.99 


.003** 


16.36(21.59) 


9.28 (8.24) 


4.44 


.000** 



♦ p < 1.00 ** p < .05 
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BEST COPY AVAILABLE 



With respect to personal experience, results of the 
analyses performed indicate mixed effects. In particular, 
Table 1 shows no significant impact of experience on 
individual performance in a constrained interaction 
condition. Although participants’ decision errors tended 
to decrease slightly from earlier to later phase, the 
change was not statistically significant in either mean 
absolute error (4.06 vs 3.62, ns) or percentage error 
(19.25 vs 16.36, ns). On the other hand, Table 1 shows 
a significant positive effect of increased level of 
experience on individual performance in the encouraged 
social interaction condition. There was a significant 
decrease in participants’ decision errors from earlier to 
later phase both in terms of absolute error (3.38 vs 2.33, 
t=4.01, p< .05) and percentage error (13.56 vs 9.28, 
t=4.45, p< .05). The results indicate that participants 
tended to learn over time and improve performance 
when encouraged to interact with others. 

With respect to the simple effects of social interaction, 
results of the analyses performed indicate that interaction 
with others had a substantial beneficial impact on 
participants’ performance irrespective of experience. 
Table 2 reveals significant positive effects of social 



interaction on decision errors in both earlier and later 
phases. The m ean abso lute error of th e participan ts in the 
encouraged interaction group was significantly smaller 
than that of their counterparts in the constrained 
interaction group in earlier (3.38 vs 4.06, t=1.87, 
p<1.00), as well as later (2.33 vs 3.62, t=4.20. p<.05) 
period. Similarly, the participants in the encouraged 
interaction group had significantly smaller percentage 
errors than their counterparts in the constrained 
interaction group, both in earlier( 13.56 vs 19.25, t=2. 99, 
p<.05) and later (9.28 vs 16.36, t=4.44, p<.05) phases of 
the task. 

Results of the analysis performed are further presented 
graphically in Figure 1. In addition to error scores of 
actualparticipants by experimental conditions, the figure 
also includes the corresponding errors of notional naive 
and optimal decision makers. These are error scores that 
would have been obtained by people who produced their 
decisions by using naive and optimal strategies for task 
performance. These scores were used to assess how 
much of the theoretically possible potential of 
information to improve performance was used by the 
actual participants. 



FIGURE 1 

INDIVIDUAL DECISION PERFORMANCE BY PERSONAL EXPERIENCE 
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The analysis shows a significant difference in the mean 
absolute error scores between actual participants and 
notional naives (4.06 vs 5.18, t=2.99, p< .05) in the 
earlier phase and constrained interaction condition. 
There was also a slight, but not significant difference in 
the mean percentage error scores (21.68 vs. 19.25, 
t=1.23,ns). The participantstendedto make smallerthan 
naive decision errors as a result of using some of the 
potential of their available information to improve 
performance irrespective of experience and interaction 
condition. 

The analysis furthershowsasignificantdifference in the 
mean absolute error scores between actual participants 
and notional optimals (2.3 3 vs 0.87, t=9.87, p< .05) in 
the later phase and encouraged interaction condition. , 
There was also a significant difference in the mean 
percentage error scores (9.28 vs 3.56, t=9.31, p< .05). 
The participants tended to make significantly greater 
decision errors than they theoretically could have with 
the available information, irrespective of experience and 
interaction condition. 

DISCUSSION 

The findings of this study provide qualified support for 
the proposition that a knowledge management initiative 
such as providing an opportunity for independent 
experiential learning will enhance individual working 
knowledge and improve performance in a non- 
deterministic decision making task context. Beneficial 
effect of greater experience was evident only in the 
interactive environment. A knowledge management 
initiative aimed at encouraging information sharing 
through social interaction was usefulness and enhanced 
performance irrespective of experience. 

With respect to personal experience, the findings of the 
current study indicate thatits impacton performance was 
contingent upon social interaction. Contrary to 
theoretical expectations, the current study found no 
significantimprovement in individu al perform ance with 
experience in the non-in teractive environm ent. Although 
participants in the non-interactive group tended to make 
slightly smaller decision errors in the later than in the 
earlierphase of the task, the change was not statistically 
significant. The lack of significant learning and 
performance improvement found in the non-interactive 
decision making environment is not surprising. It is 
consistent with a large body of knowledge accumulated 
from judgement and decision making research involving 



non-deterministic multi-variate tasks with or without 
feedback (for review see Brehmer, 1980). 

It seems that the history of past errors expected to help 
learning (Kleiner & Roth, 1998) could not provide the 
participants with the information as to what works and 
what does not in the context of a non-deterministic task. 
The same decision error could be produced by many 
different misapprehensions about the relations between 
the task variables. According to Brehmer (1980) people 
can not learn probabilistic relations because they assume 
that the variables are related by deterministic rule. 
Howeve r, the fact that the particip ants responded in the 
appropriate direction suggests that the y could po tentially 
achieve significant improvement if given more trials. 
The results from the cue discovery tasks (Klayman, 
1988) indicate that it was possible to accomplish cue 
discovery gradually over a larger number of trials 
(ranging into the hundreds). 

Contrary to most previous research, the currentstudy has 
demonstrated a substantialbeneficial effect of increased 
experience on decision accuracy in the interactive 
decision making environment. Participants who 
interacted with others while handling the decision 
problem tendedto exhibitsignificant learning over time 
which led to improved quality of their decisions. This 
was demonstrated by significantly smaller absolute and 
percentage errors found among these participants in the 
later than in the earlier period of the task This is an 
important finding. It suggests that social interaction 
enhanced learning. These participants were able to learn 
better how to us e their available information to improve 
performance. As a result, they tended to make more 
accurate decisions. It is possible that the opportunity to 
discuss various aspectsof the task with othershelped the 
participants better assess the quality of the available 
information and evaluate their tentative prediction 
strategies. It could also help them perceive the task as 
less complex. 

Indeed, the current study has demonstrated that social 
interaction had a beneficial effect on individual 
performance irrespective of experience. More 
specifically, encouraged social interaction led to 
improved decision accuracy in both earlier and later 
stages of the task. When participants were allowed to 
interact and share ideas with others while handling the 
decision problem, they were found to make smaller 
decision errors (absolute and percentage) than when they 
performed the same task without such interaction. This 
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was true in both less and more experienced stages of the 
task. 

The beneficial effect of social interaction evidentin this 
study is consistent with the theoretical expectations 
suggested by the knowledge management literature 
(Garvin, 1998; Nonaka & Takeuchi, 1 995), It also agrees 
with numerous anecdotal evidence from the real world 
organisations (Hewson, 1999), However, the results 
obtained in this study contradict findings by Heath and 
Gonzalez (1995) who investigated the issue in the 
similarly complex predictive task domain in study 2, The 
difference between the two findings can be potentially 
attributed to the characteristics of the task context in 
which the investigation was carried out. Different from 
Heath and Gonzalez, the current study provided 
participants with immediate performance feedback. 
Earlier empirical research indicated beneficial effect of 
feedback on learning (Kopelman, 1986), 

Participants might have brought theirpersonal analysis 
and know-how to the task, acquired information about 
their partner’s ideas and argum ents and co nsidered b oth 
in making final decisions. Immediate performance 
feedback might have enabled participants to evaluate 
their own ideas against those of their partners or jointly 
generated ones, and adjustfuture strategies accordingly. 
Generating and sharing personal tacit knowledge through 
interaction coupled with the opportunity to test its 
contribution to performance over time might have 
enhanced learning and resulted in greater accuracy. 

The current study also tried to avoid a potential adverse 
effect of advisor-conflict by limiting individual’s social 
interaction to one other person. Sniezek and Buckley 
(1995) reported that in a two-advisor system with 
conflict, advice did not improve judge’s accuracy and 
resulted in deteriorated performance in some cases. In 
addition, the participants in the current study had dual 
judge-advisor roles. Because of their equal status in the 
interactive act, it is possible that the participants jointly 
generated some new ideas previously non-existent in 
their individual contexts. Previous research indicated 
that teams of two people were successful in performing 
collaborative activities (Panko & Kinney, 1992; 
Schwartz, 199 5). 

With respect to overall performance, the study revealed 
significant improvement in decision accuracy over naive 
strategy across all treatments. Thiswas demonstrated by 
significantly smaller decision errors found among actual 
participants than their notional naive counterparts. The 



results suggest that people need very little experience 
(eg. just an initial short practice) w ith the task to be able 
to recognise relevant and valid information as such. As 
a result they are able to use some of its potential to 
improve performance. This agrees with some earlier 
findings (Klayman, 1988) indicating that while people 
have difficulties in learning its shape, they are generally 
able to perceive the existence and direction of a cue- 
criterion relation. 

Unfortunately, the study revealed significantdeviations 
from optimal performance across all treatments. 
Participants tended to use much less offull information 
potential and perform much worse than they could have. 
This was demonstrated by significantly greater decision 
errors found among actual participants than their 
notional optimal counterparts. The suboptimal overall 
performance could be potentially attributed to the lack of 
monetary incentives. Sniezek and Buckley (1995) 
provided their students with substantial monetary 
rewards for their performance. It is possible that without 
monetary incentives, the participants did not try as hard 
as possible to use more potential of their information to 
improve decisions. By not giving extrinsic incentives, 
this study attempted to prevent information “hoarding , 
and promote cooperationrather than competition among 
the participants. It was also assumed that graduates 
chosen from the pool of students attending an advanced 
level course should be motivated to do their best on the 
task by the intrinsic interest for the subject matter. 

Alternatively, the failure to a chieve op timal p erformance 
could be attributed to the characteristics of the task 
information. Information repositories available to 
participants in the current study contained only factual 
information with little analysis, and had no procedural 
information. Acquiring “explicit factualknowledge was 
not sufficient to perform well on the task. The 
participants needed also analytical information such as 
evaluation of predictive validity of each contextual 
factor, as well as the relevant know-how to integrate 
factual information into a decision response. This crucial 
information was assumed to be a part of individual 
“tacit knowledge acquired from experience and social 
interaction. However, it is possible that non-expert 
participants needed to either interact with experts or be 
given more time to develop higher levels of the required 
“tacit knowledge to perform well on the task. 

While the currentstudy provides a number of interesting 
findings, some caution is necessary regarding their 
generalisability due to a number of limiting aspects. One 
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of the limitations refers to the use of a laboratory 
experiment that may compromise external validity of 
research. Another limitation relates to artificial 
generation of information that may not reflect the true 
nature of real business. The participants chosen for the 
study were students and not real life decision makers. 
The fact that they were mature graduates may mitigate 
the potential differences. No incentives were offered to 
the participants for their effort in the study. 
Consequently, they could find the study tiring and 
unimportant and would not try as hard as possible. Most 
decisions in real business settings have significant 
consequences. 

Although limited, the findings of the current study may 
have some important implications for organisational 
knowledge management strategies. They indicate that 
novice knowledge workers left on their own in working 
environments that do not encourage culture of sharing 
are- not likely to learn effectively from experience 
(beyond the initial learning through short practice) and 
are not likely to further improve performance. Creating 
working contexts that encourage communication and 
culture of information sharing among knowledge 
workers may potentially have a beneficial effect on 
knowledge creation by speeding up individual learning 
and resulting in enhanced performance. The findings 
may also have some important pedagogical implications 
for distance education, given the growing number of 
distance learning programmes offered by various 
university colleges and business schools. In particular, 
the evidence from this study suggests that support 
facilitiessuch as groupware systems may be necessary to 
enable interaction among distance learners to augment 
their individual experience. 

CONCLUSIONS 

The main objective of this study was to investigate the 
effects of personal experience and social interaction on 
people’s working knowledge and use of that knowledge 
to improve performance in a specific decision making 
context. In summary, the findings of the study indicate 
that increased experience did not result in significant 
performance improvement of individual decision makers 
when no opportunity for interaction with others was 
available. In the interactive environment, increased 
experience led to enhanced decision accuracy. The 
findings further indicate a beneficial effect of social 
interaction on performance irrespective of decision 
makers’ experience. The encouraged social interaction 



was useful at both low and high experience stages of the 
task and resulted in enhanced decision accuracy when 
compared to constrainedinteraction. These findingsmay 
have important implications for knowledge management, 
as they reveal synergy of the two combined factors in 
effective knowledge creation and subsequent 
performance improvements. Furtherresearch is required 
that would extend the study to other participants and 
environmental conditions in order to ensure the 
generalisability of the present findings. 

REFERENCES 

Arkes, H.R., Dawes, R.M. & Christensen, C. (1986). 
Factors Influencin g the Use of a Decision Rule in a 
Probabilistic Task. Organisational Behaviour and 
Human Decision Processes, 37, 93-1 10. 

Arthur Andersen Office of Training and Education 
(AAOTE) (1998). BC Knowledge Management. 
Arthur Andersen. 

Ashton, R.H. & Kramer, S.S. (1980). Students as 
Surrogates in Behavioural Accounting Research: 
Some Evidence. Journal of Accounting Research, 
18(1), 1-15. 

Balzer, W.K., Hammer, L.B., Sumner, K.E., 
Birchenough, T.R., Martens, S.P. & Raymark, P.H. 
(1994). Effects of Cognitive Feedback Components, 
Display Format, and Elaboration on Performance. 
Organisational Behaviour and Human Decision 
Processes, 58, 369-385. 

Balzer, W.K., Sulsky L.M., Hammer, L.B. & Sumner, 
K.E. (1992). Task Information, Cognitive 
Information, or Functional Validity Information: 
Which Components of Cognitive Feedback Affect 
Performance? Organisational Behaviour and 
Human Decision Processes, 58, 369-385. 

Baxter, J. & Chua, W.F. (1999). Now and the Future. 
Australian Accounting Review, 9(3), 3-14. 

Brehmer, B. (1980). In One Word: Not from Experience. 
Acta Psychologica, 45, 223-241. 

Connolly, T. & Gilani, N. (1 982). Infonnation Search in 
Judgement Tasks: A Regression Model and Some 
Preliminary Findings. Organisational Behaviour 
and Human Performance, 30, 330-350. 



46 



Proceedings of the 15^ Annual Conference of the International Academy for Information Managemait 



Connolly, T. & Serre, P. (1984). Information Search in 
Judgement Tasks: The Effects of Unequal Cue 
Validity and Cost. Organisational Behaviour and 
Human Performance^ 34, 387-401. 

Connolly, T. & Thorn, B.K. (1987). Predecisional 
Information Acquisition: Effects of Task Variables 
on Subop timal Search Strategies. Organisational 
Behaviourand Human Decision Processes^ 39, 397- 
416. 

Connolly, T. & Wholey, D.R. (1988). Information 
Mispurchase in Judgement Tasks: A Task Driven 
Causal Mechanism. Organisational Behaviour and 
Human Decision Processes, 42, 75-87. 

Creyer, E.H., Bettman, J.R. & Payne, J.W. (1990). The 
Impact of Accuracy and Effort Feedback and Goals 
on Adaptive Decision Behaviour. Journal of 
Behavioural Decision Making, 3(1), 1-16. 

Davenport, T.H. & Prusak, L. (1998). Working 
Knowledge, Boston: HarvardBusiness School Press. 

Davenport, T.H., DeLong, D .W. & Breers, M.C. (1998). 
Successful Knowledge Managem ent Projects. Sloan 
Management Review, Winter, 43-57. 

Devlin, K. (1999). Infosense . Turning Information into 
Knowledge. New York: W.H. Freeman and 

Company. 

Drucker, P.F. (1993). Post-Capitalist Society, New 
York: Harper Business. 

Drucker, P.F. (1998). The Coming of the New 
Organisation. Harvard Business Review on 
Knowledge Management, Boston: Harvard Business 
School Press. 

Garvin, D.A. (1998). Building a Learning Organisation. 
Harvard Business Review on Knowledge 
Management. Boston: Harvard Business School 
Press. 

Grayson, C.J. & Dell, C.O. (199 8). Mining Your Hidden 
Resources. Across the Board, 35(4), 23-28. 

Hastie, R. (1986). Experimental Evidence on Group 
Accuracy. In Grofman B. & Owen G. (eds) Decision 
Research, 2, 129-1 57. 



Heath, C. & Gonzalez, R. (1995). Interaction with 
Others Increases Decision Confidence but Not 
Decision Quality: Evidence against Information 
Collection Views of Interactive Decision Making. 
Organisational Behaviour and Human Decision 
Processes, 61(3), 305-326. 

Hewson, D. (1999). Start Talking and Get to Work. 
Business Life, November, 72-76. 

Hogarth, R.M. (1981). Beyond Discrete Biases: 
Functional and Dysfunctional Aspects of 
Judgemental Heuristics. Psychological Bulletin, 
90(2), 197-217. 

Janis, I (1982). Groupth ink: Psychological Studies of 
Policy Decisions and Fiascoes, Boston: Houghton- 
Miffin . 

Klayman, J. (1988). Learning from Experience. In 
Brehmer, B. & Joyce, C.R.B. (eds) Human 
Judgement, The SJT View, Amsterdam: North- 
Holland. 

Kleiner, A. & Roth,G. (1998). How to Make Experience 
Your Company’s Best Teacher. Harvard Business 
Review on Knowledge Management. Boston: 
Harvard Business School Press. 

Kopelman, R.E. (1986). Objective Feedback. In Locke, 
E.A. (ed) Generalising from Laboratory to Field 
Settings, Lexington Books. 

Lim, J.S. & O'Connor, M.J. (1996). Judgemental 
Forecasting with Interactive Forecasting Support 
Systems. Decision Support Systems, 16, 339-357. 

Makridakis, S. (1993). Accuracy measures: theoretical 
and practical concerns. International Journal of 
Forecasting, 9, 527-529. 

Makridakis, S. & Wheelwright, S.C . (1989). Forecas ting 
Methods for Management, 5th ed. New York: John 
Wiley and Sons. 

Marakas, G.M . (1999). Decision Support Systems in the 
21st Century. New Jersey: Prentice Hall Inc. 

Nonaka, I. (1998). The Knowledge-Creating Company. 
Harvard Business Review on Knowledge 
Management. Boston: Harvard Business School 
Press. 



Proceedings of the 1 5*^ Annual Conference of the International Academy for Information Management 47 



57 



BEST COPY AVAILABLE 



Nonaka, I. & Takeuchi, H. (1995). The Knowledge 
Creating Company: How Japanese Companies 
Create the Dynamics of Innovation, New York: 
Oxford University Press. 

O'Connor, M.J. (1989). Models of Human Behaviour 
and Confidence in Judgement: A Review. 

InternationalJournal of Forecastings 5, 159-169. 

Panko, R. & Kinney, S. (1992). Dyadic Organisation 
Communication:Is the Dyad Different? Proceedings 
of the 25th Hawaii International Conference on 
Systems ScienceSs 244-253. 

Payne, J.W., Bettman, J.R. & Johnson, E.J. (1988). 
Adaptive Strategy Selection in Decision Making. 
Journal of Experimental Psychology: Learning, 
Memory and Cognition^ 14(3), 534-552. 

Peterson, D.K. & Pitz, G.F. (1988). Confidence, 
Uncertainty and the Use of Information. Journal of 
Experimental Psychology: Learning, Memory and 
Cognitions 14, 85-92. 

Remus, W. (1996). Will Behavioural Research on 
Managerial Decision Making Generalise to 
Managers? Managerial and Decision EconomicSs 
17, 93-101. 

Remus, W., O’Connor, M. & Griggs, K. (1 996). Does 
Feedback Improve the Accuracy of Recurrent 
Judgemental Forecasts, Working paper, University 
of Hawaii, January. 

Sauer, C. (1993). Why Information Systems Fail: A Case 
Study Approach, Henley-on-Thames: Alfred Waller. 



Schwartz, D. (1995). The Emergence of abstract 
representation in dyad problem solving. The Journal 
of the Learning ScienceSs 4(3), 32 1-345 . 

Senge, P. (1990). The Fifth Discipline, New York: 
Double-day. 

Sniezek, J. A & Buckley, T (1 995). Cueing and 
Cognitive Conflict in Judge-Advisor Decision 
Making. Organisational Behaviour and Human 
Decision ProcesseSs 62(2), 159-174. 

Stasser, G., Taylor, L.A., & Hanna, C. (1989). 
Information Sampling in Structured and 
Unstructured discussions of three- and six-person 
groups. Journal of Personality and Social 
Psychologys 57, 67-68. 

Stewart, T.A. (1997). Intellectual Capital: The New 
Wealth of Organisations, New York: Doubleday. 

Swanson, E.B. (198 8). Information Systems 
Implementation, Homewood: Irwin. 

Whitecotton, S.M. (1996). The Effects of Experience 
and a Decision Aid on the Slope, Scatter, and Bias 
of Earnings Forecasts. Organisational Behaviour 
and Human Decision ProcesseSs 66(1), 1 11-121. 

Yates, J.F. (1991). JDM conference presentation, 
November, San Francisco. 



48 



Proceedings of the 15*^ Annual Conference of the International Academy for Information Management 



THE LEARNING EFFECTIVENESS OF 
INSTRUCTIONAL TECHNOLOGIES: 
RESULTS FROM PILOT STUDIES 



Katia Passerini 

The George W ashington University 



Mary J. Granger 

The George Washington University 



ABSTRACT 

This paper res ponds to c alls for further research on learning effectiveness of instructional technologies by comparing 
'learning'and 'attitudes' across thre e instruction al environments (multimedia, textbook and in-class instruction), and 
proposing a quasi-experimental research design approach. The paper reports the results of pilot studies conducted 
with the aim of refining the research framework, assessing the experimental conditions, and obtaining preliminary 
results. This research framework and the pilot studies are described. 



INTRODUCTION 

Since technology is so widespread in society, different 
stakeholders believe that educational strategies should 
reflect this expanding phenomenon by introducing 
instructional technology in the classroom. They often 
assume that the effectiveness of technology has been 
proven once and for all (Roblyer et al, 1997). Instead, 
the learning effectiveness of instruction al technology is 
yet to be proven. In spite of the several studies (Nugent, 
1982; Mayer & Anderson,! 991) and meta-analyses 
(Kulik, Kulik & Shwalb, 1986; Roblyer, Castine & 
King, 1988; Kulik, 1994; Khalili & Shashaani, 1994;) 
competing to answer the question: “do computers and 
other related technologies make a difference in 
learning? the question on the comparative performance 
of instructional technology is notyet answered. Several 
scholars are calling for further research (Liao, 1998; 
Jones & Paolucci, 1998; Fletcher-Flinn & Gravatt, 
1995). 



The authors try to answer the call for further research 
with a quasi-experimental design which compares 
'learning' and 'attitudes' (satisfaction and appeal) in 
subjects gaining knowledge of project management 
topics by either attending a class presentation, reading a 
textbook, or using an interactive multimedia CD-ROM. 
The subjects of th e study are graduate a nd unde rgraduate 
students enrolled in business administration degree 
programs at a major East-coast University . The study is 
replicated, under comparable conditions, with the two 
groups. The purpose of the replication is to explore 
whether learning effectiveness and satisfaction vary by 
age (graduate and undergraduate), and by subject topic 
characteristics. 

Because the results from the implementation of the 
quasi-experiment are not available at the present time, 
this paper focuses in describing the theoretical 
framework, the research design, and the results of the 
pilot studies that precede the actual testing. The results 
from the analysis wiD be available in December 2000. 
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THE LEARNING EFFECTIVENESS OF 
INTERACTIVE MULTIMEDIA SYSTEMS 

The authors' expectations based on the literature review 
is that the instructional conditions that best support 
multiple representation modes have a higher impact on 
learningspecific content. Instructional technologies such 
as interactive multimedia allow the integration of 
multiple media in a specific delivery environment, 
therefore promoting multiple representations of 
knowledge. The structural features (organization of 
content, modes of delivery) of interactive multimedia 
have an impact in the construction of mental models 
(Jonassen, 1990) and, particularly, a positive effect on 
the cognitive system. 

Tergan (1997) states that the cognitive processing of 
multiple external representations (such as the ones in 
interactive multimedia and hypermedia) matches 
principles of encoding variability and encoding 
specificity (Tulving, 1983). The encoding specificity 
principle requires that learning situations contain 
recognizable clues thathelp learnersunderstand, locate, 
and recall information; and that realistic contexts (like 
the ones in interactive multimedia systems) generally 
result in better performance. The principle of encoding 
variability is also supported by interactive mltimedia 
multiple representation modes. The fact that information 
is encoded in a variety of formats increases the chances 
that at least one ofthe representations correspondsto the 
context of use in a particular situation (Hammond, 
1993). 

The value of interactive multimedia learning is also 
supportedby the principles of cognitive flexibility theory 
(Jacobson & Spiro, 1995); multimodal cognitive 
processing (Cunninhgham et al., 1993); and indi- 
vidualized and self-regulated instruction (Duffy & 
Knuth, 1990). According to the cognitive fiexibility 
theory, interactive multimedia environments employ 
multiple ways to represent knowledge in instructional 
activities, thus reflecting the nature of complex 
knowledge. This contributes to the development of 
cognitive structures that are flexible enough to be 
applicable to different contexts, enhancing knowledge 
transfer. Paivio (1986) finds that using visuals and other 
representational modes entails dual coding, which 
enables humans to better retain information. Paivio ’s 
theory is supported by recent developments of 
multimedia approaches to learning (Engelkamp & 



Zimmer, 1994). Remembering, for example, may be 
enhancedbecause information is represented in different 
cognitive systems. 

The learning effectiveness argument for multimedia is 
also supported by the multiple intelligences theory 
(^Gardner, 1984, \ 993) 2 n\demotional intelligence theory 
(Goleman, 1995). Multiple symbolic modes promote 
synergistic interactions among several intelligences 
(spatial, kinesthetic, logical-mathematical, musical, 
linguistic, interpersonal, intrapersonal) and benefit 
learning in the various domains - cognitive, affective, 
psychomotor - (Kemp, 1994). Computer-based media 
support the social dimensions of human learning, 
interpersonal and intrapersonal intelligences, to an 
astonishing degree (McLellan, 1996). 

By affecting different domains (cognitive, affective and 
psychomotor) multimedia also allows mass- 
customization of instruction, since instruction offered in 
several modes is adaptable to the individual learner. 
Different learners are more attuned to different types of 
representations (Keller, 1983) and implementing 
reinforcing activities may enable them to “explicitly or 
“implicitly selectthe activity that is most “customized 
to their cognitive skills (McLellan, 1 996). In addition, by 
shifting between different representations, learners get 
multiple practice (M arks-Tarlow, 1995). 

RESEARCH DESIGN 

In this framework, the main question ofthe researchers 
is 'Whether interactive multimedia applications are 
effectivelearning toolscompared to traditionallearning 
environments (textbook, or in-class instruction)/ To test 
the learningeffectiveness ofinteractive multimedia, this 
study adopts the definition of learning in MerrilTs 
performance-content matrix model (1983). Based on this 
definition, the independent variables* (the three 
instructional settings, such as the class presentation, the 
textbook, and the multimedia CD-ROM) effects on the 
learning of project management topics {dependent 
variable) are examined through comparing subjects’ 
recall and application performance in specific content 
areas (recall and application of facts, concepts, 
principles, and procedures). Subjects’ attitudes 
{dependent variable) across the instructional settings are 
investigated through learners’ comparative satisfaction 
with the instmctional materials (see Figure 1). 
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FIGURE 1 

QUASI-EXPERIMENTAL MODEL FOR TESTING THE 
COMPARATIVE EFFECTIVENESS OF INSTRUCTIONAL TECHNOLOGIES 




The intervening variables include: student 

characteristics such as gender, age, prior knowledge, 
computer abilities, and learningpreferences; and, subject 
characteristics, such as complexity of the topic (high 
and low complexity). Assessing learners’ prior 
knowledge of the topic and reporting the ir demographic 
breakdown reinforces the research design. It enables 
comparing and contrasting the treatment and control 
groups, as well as developing a better understanding of 
the characteristics of the students surveyed. 

Because the effectiveness of the instructional materials 
may also depend on learners’ prior experience in 
management, the study utilizes a pre-test & posttest 
design to assess learners prior knowledge of the topic. In 
addition, the demographic survey collects data on 
learners prior work experience (see, for example, F/gw re 
2), major, and computer utilization. 

Related questions associated to the model presented in 
Figure 1 and based on Merrill’s (1983) performance- 
content matrix include: 



Which environment is more effective at achieving 
learning objectives (recall or application)? 

Which environment is more effective at delivering 
a specific type of topic (with high/low complexity)? 

Which environment, if any, is more appealing for 
the learners’ groups (graduate, undergraduate) 
targeted? 

The responses to these questions based on the 
implementation of the study will be available in 
December 2000. Although it is expected that it will be 
difficult.to make clear-cut and distinct statements on the 
instructional effectiveness of technology (interactive 
multimedia may be more effective for specific content 
areas, such as recall oT facts and principles, but less 
effective in applications of procedures), the authors set 
up a study that tries to overcome the limitations 
recognized in earlier studies. 
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DIFFICULTIES AND REMEDIES 

There are many factors that can influence the results of 
effectiveness studies. The researchers recognize the 
difficulty and address the objections brought to similar 
comparative learning research. In particular, Clark 
(1983) offered many objections to the value of the 
analyses that claimed comparatively positive outcomes 
in educational technologies. According to Clark (1983) 
the most common sources of confounding evidence in 
earlierstudies appeared to be the uncontrolled effects of: 
a) instructional method or content differences between 
treatments that are compared; and b) novelty effect for 
newer media, which tends to disappear over time. 

1. The research design of this study addresses Clark’s 
considerations and tries to isolate learning from the 
medium by limiting the instructional time, 
controllingthe learning environment, and assessing 
the role of intervening variables. 

2. To address Clark’s first concern (point a), the 
researchers focus on one specific technology 
(multimedia) and compare its effectiveness with 
both classroom and text-based instruction in 
teaching project management modules using the 
same teaching materials. The characteristics of the 
instructional material are briefly described to 
provide support to the comparability statement. 

The Textbooks 

The textbooks from which the multimedia CD- 
ROM, and the in-class instmction, are based are two 
major project management publications. 

Frame (1994) The New Project 
Management: Tools for an Age of Rapid 
Change, Corporate Reengineering, and 
Other Business Realities. The Jossey-Bass 
Management Series 

Frame (1995) Managing Projects in 
Organizations: How to Make the Best Use 
of Time, Techniques, and People. The 
Jossey-Bass Management Series 

Both publications are ranked highly (number of 
sales, and reviewers’ comments) in the project 
management literature. They both present similar 
design and layout features, and they both address 
complementary topics. They use diagrams and 



drawings to reinforce understanding, offer several 
examples, and occasionally include mini-case 
studiesto foster reflection and application. The case 
studies present problems, and solutions, to emulate 
a mechanism of feedback provision. 

The multimedia CD-ROM 

The ‘Project Management in Organizations 
(PMO) CD-ROM’ is the multimedia version of 
Frame, 1994 & Frame, 1995. The CD-ROM 
closely follows the books’ organization: 
learningunits mirror chapter titles andstructure. 
The application is designed to teach the 
fundamentals of project management. 
Audiences vary from aspiringproject managers, 
project team members, executives, or other 
stakeholders interested in learning project 
man agement. 

The CD-ROM provides a stand-alone tool for 
learningprojectmanagement introductory topics 
and is copyrighted by the Education Services 
Institute - ESI-International, 1997. It includes 
different media formats and features: 

full motion video 

audio files 

text-based animation 

extensive interactivity (with case 

studies and other exercises) 

high-quality graphics 

user-friendly navigation interface 

It also offers a Toolbox with examples of tools, 
access to Resources (calculator, glossary, 
notepad), and an Organizer, to sort information 
in the program, and access testing. 

In-class instruction 

The lecture presentationsby J. DavidsonFrame 
are supported by the use of overhead 
transparencies. The transparencies follow the 
organization of Frame, 1994, 1995. They 
display the graphical images of the textbook, 
and develop additional graphical 
representations. The instructor primarily uses 
markers to write on transparencies during the 
presentation. Occasionally, display boards are 
used to support explanations. 
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The instructor offers several examples, asks 
frequent questions to the audience, provides 
feedback to responses, and encourages 
participation through in-class discussion of 
section problems. Frequently, cases and other 
exercises are completed in-class, with the 
instructor support. 

3. The study does not address the issue of “novelty. 
Clark’s point b) suggests that the use of technology 
has an initial positive impact on learning, but this 
impact is lowered over time (what Clark calls 
'‘"novelty effect). Clark himself (Clark & Sugrue, 
1995) reports that some studies on the use of 
computers in college (Kulik & Kulik, 1 980) did not 
find any evidence for the novelty effect. Kulik and 
Kulik’s comparisons of studies of one or two hours 
duration, vs. studies which held weekly sessionsfor 
an entire semester, reported that the effect sizes on 
learning were roughly the same. Clark responds to 
these findings by pointing to the fact that the use of 
computers may be a less “novel experience for 
college students than for other younger students 
(p.354). 

Because the subjects of this research are college 
students, following Kuliks’ findings and Clark’s own 
considerations, this study limits the length of the 
instructional modules to short-segments of instruction 
(from 1 hour and 15 minutes to 2 hours of instruction). 
Moreover, the limited length ofthe instructionalmodules 
enables focusing on the impact of the medium on 
achievement and satisfaction rate by eliminating 
competing variables (such as “total study time ) that 
studies with delayed post-testing may introduce. The 
researchers recognize that the use of short-instructional 
modulesintroduces some limitations regarding the types 
of knowledge and performance tasks that are achievable 
within the limited timeframe. 

METHODOLOGY AND FOCUS 
REFINEMENT THROUGH PILOT STUDIES 

Three pilot studies precede the refinement of both the 
theoretical framework and the procedures described in 
this study. They are valuable elements of the quasi- 
experimental approach and are, therefore, reported in 
this paper. 



Research Methodology Refinement - Pilot Study (1) 

A first pilot study has been instrumental to refine the 
methodological approach (and deciding between a short- 
term approach vs. a longer-term study). The researchers’ 
aim to exploit the advantages of the media by offering 
participants the opportunity to use the textbook/s, or the 
CD-ROM at their own timing, outside of a laboratory 
environment, and with little control of the instructional 
conditions has been prevented by a high-mortality rate. 

The sample population for this first pilot designed to 
assess the feasibility of a long-term self-paced study was 
composed of working professionals registered to 
monthly workshops in Project Management offered by 
a major project management educational services 
institution in the US. Individuals registered to the 
workshop received an invitation to participate in the 
study. Volunteers were randomly assigned to two 
treatments (interactive multimedia CD-ROM or 
textbook-instruction). Individuals registered in the 
workshop were assigned to the lecture treatment. They 
completed paper-and-pencil surveys. Volunteers in the 
other treatment groups took all the surveys on-line. 

The outcomes of this pilot study supported the need of 
controlling for instructional conditions, thus undertaking 
the study in a real-laboratory setting. Although 
interactive multimedia applications are designed to 
support self-paced instruction, learners need to receive 
constant reinforcement and feedback to complete the 
training program. Although over 23% of the 100- 
individuals contacted volunteered their participation, 
13% dropped prior to receiving the instructional 
materials. Of the remaining 10% who received the 
instructional materials, only 5 people completed the 
study on time (prior to their scheduledtraining session). 
The high mortality rate has been instrumental in 
redesigning the research to test, only in a controlled 
setting, graduate and undergraduate groups enrolled in 
business administration degree programs. 

Demographic Sample - Pilot Study (2) 

Another pilot study was conducted with the objective of 
testingscale reliability and assessesthe comparability of 
MBA-Cohort groups. The pilot was held during the 
laboratory sessions of two sections of an MBA-Cohort 
Information System course. Students enrolled in 
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Tuesday and Wednesday’s laboratory sessions (hereafter 
referred to as Tuesday, or Wednesday students) took an 
on-line questionnaire on project management topics 
ranging from Time Management, Cost Management, 
Human Resources Management, Risk Management, 
Communication Management, and Scope Management. 
They also completed a demographic survey. 

The purpose of the demographic survey was to compare 
the characteristics of the Tuesday and Wednesday 
groups to identify any specific group composition bias, 
which could inhibitthe generalizability of the findings, 
and caution against the use of the scale in the actual 
study. The demographic data evaluated group variances. 
The frequency results of the demographic surveys 



confirmed that difference between the two groups 
(Tuesday and Wednesday) in this population are 
minim al: 

65.5% ofthe Tuesday participants are Asians/Pacific 
Islanders, 64.5% of the students in the Wednesday 
group report belongingto the same ethnicity group. 
18.8% (T) vs. 26.9% (W) are CaucasiansAVhite; 
75% (T) vs. 81.5% (W) have a college degree, 
21.9% (T) vs. 14.8% (W) have a masters degree; 
15.6% (T) vs. 14.8% (W) work in a management 
field, 68.8% (T) vs. 70.4% (W) are in the education 
field (mostly as students); - sqq Figure 2 - 
Additionally: 96.9% (T) vs. 92.6% (W) are between 
20 and 29 years old - See Figure 3 - 



FIGURE 2 

PRIOR WORK EXPERIENCE 
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FIGURE 3 

AGE OF THE LEARNERS 



Wednesday Group Demographics 
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Tuesday Group Demographics 
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43.8% (T) vs. 55.6% (W) report of having been using a computer for 7 years or more; 

50% (T) vs. 59.3% (W) are male; 

37.5% (T) vs. 40.7% (W) report of having used the computer primarily at work/school, but also at home. 



The frequencies values reported above display ranges of 
differences from 0.9% to less than 10% difference on 
same measures, with the mode of occurrences 
concentrating on the lower differences. Because more 
recent assignments of students to cohort groups have 
standardized stratification mechanisms, it is expected 
thatsimilarities in the groups are further increased in the 
groups participating in the redesigned study. 

Satisfaction & Attitudes -Pilot Study (3) 

A pilot test of the undergraduate students (senior level) 
was undertaken in one section of a ‘Change in 
Organizations’ course. The main objective of the pilot 
was to collect feedback on undergraduate students’ 
appreciation of the selected project management topic, 
and their attitudes toward the experimental procedures 
(i.e. special attention focuses on their reactions to the 
presentations, and their interest in attending another 
session on the topic). Because the sample size of the 
pilot group was small (44 students present in the 
sessions), and their subgroups were composed of only 1 5 



or less students, the results from the statistical analysis 
are not reliable, and are omitted in this discussion. 
Interesting findings, however, emerge from the 
descriptive frequencies analysis of satisfaction, and 
appeal measures (see Figure 5 and Figure 6). 

Figure 4 shows that students in the lecture group (#1) 
were the most satisfied with the organization, pace, 
content of instruction, and with the instructor’s 
presentation (means > than 4.5). The other groups’ 
satisfaction was also above the mid-point (mean = 4), 
with the CD-ROM group being more satisfied than the 
textbook group with the organization of the materials 
(ORGANIZ). At the same time, the CD-ROM group was 
the least satisfied in terms of content, pace, and 
presentation (FRAME). Although the size of each 
difference was within a 1 -point difference, the CD-ROM 
participants were consistently least satisfied with the 
materials, and felt that they did not learn the material 
satisfactorily (LEARN). 
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FIGURE 4 

SATISFACTION SURVEYDATA, 
UNDERGRADUATE PILOT STUDY, FALL 1999 
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It is interesting to compare the values in Figure4 with 
other results. Students in the CD-ROM group (#2) 
reported that they had the least prior knowledge of the 
topic (Figure 5a), but they obtained the highest pretest 
and posttest mean scores (Figure 5b), with the 
limitations that the sample size is not sufficient to use 
formal parametric procedures, as well as draw 
conclusions from comparative differences. 

FIGURE 5 

PRIOR KNOWLEDGE OF THE TOPIC 
Figure 5a: Self-reported Prior Knowledge 
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Additionally, 30% of the students answered that they 
would like to attend another instructional session similar 
to the one in which they participated; and 41 % answered 
thatthey ‘may’ prefer attending asimilar session (Figure 
6 ). 



Figure 5b: Pre-TEST — Posttest Differences 
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FIGURE 6 

APPEAL RESPONSE, UNDERGRADUATE 
PILOT STUDY, FALL 1999 
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The appeal measure is encouraging, especially if 
contrasted with the conditions of the pilot study. 
Students in this pilot were substantially rushed into the 
instructional materials because of the logistic 
arrangements, the lack of proximity of the laboratory and 
the reading facilities, the need to perform other 
miscellaneous administrative tasks, substantially 
reducing instructional time. The CD-ROM group did not 
complete all the assigned sessions, and spent only a total 
of 17-minutes using the application. Similar time- 
constraints characterized the textbook-group; whereas 
the in-class group is the only group that used the time 
satisfactorily. 

The conclusions of this pilot (#3) led to an improvement 
in scheduling efforts, with each group meeting directly 
in the classroom in which the treatment is offered, and 
reassured the researcher of the appropriateness of using 
the ‘change control’ topic with this group of students. 
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Lesson learned from this, and the other pilots, guided the 
implementation of better-enforced laboratory 
procedures. 

CONCLUSIONS FROM THE PILOT STUDIES 

The implementation of the pilot studies has been critical 
to the design of an improved methodology. First of all, 
it has demonstrated that the ambition to conduct such 
experiments outside controlled lab environments is 
tainted with the inability to controlthe learningprocess. 
It results in high-mortality due to conflicting tasks and, 
above all, lack of reliability of self-reported indication 
on how/for-how-long the CD-ROM was used. The 
evaluation of the comparative effectiveness can be 
conducted only if both the instructional material and the 
instructional conditions are equivalent (controlled 
laboratory setting). This calls forthe application ofthese 
types of experiments only fora limited amount of time, 
unless the researcher has full control on the many 
intervening variables (study and practice time, etc.). It is 
no surprise that similar experiments have led to conclude 
that there are “no differences in learning across 
instructional materials . They did not take into account 
the study-time/efforts of the learners, which obviously 
influences success in the final exam regardless of the 
tool used to study the instructional material. 

Secondly, the pilot studies have helped assess the 
comparability of the groups being tested and refined the 
survey instruments. Thirdly, they havehelped screen out 
learners’ attitude (both graduate and undergraduate 
students) towards the study material, which is also 
another important component of students’ learning. 

VALUE OF THE STUDY 

Once if is completed and implemented with the changes 
suggested by the pilot studies herein described, this 
study will contribute to research on instructional 
technology effectiveness in several ways. 

It will offer a unique comparative situation, with 
coincidenttopic presentation across media (multimedia, 
textbook, in-class instruction). Therefore, it will enable 
the researcher to control for content-differences biases 
found in earlierstudies. The research design overcomes 
several limitations ofthese earlier studies (controlling 
for time, instructor, and content-differences), and 
presents a framework that addresses previous concerns. 



This research will contribute to the advancement of the 
debate on effectiveness of instructional technologies by 
looking at one specific medium, and its comparative 
impact. In fact, it responds to calls for further research in 
areas that have key instructional implications for 
education and training. 

By replicating this research with undergraduate and 
graduate students, with higher and lower complexity 
topics, and contrasting direction and magnitude (effect 
size), this research offers further validation procedures. 
The replication also differentiates learning and 
satisfaction by groups (undergraduate and graduate), and 
topic complexity. Statistically significant differences 
among the age groups, for example, allow the 
measurement of media effectiveness on differentlearner 
populations. Learning goals may not be achieved 
because the learner’s developmental stage (Piaget & 
Inhelder, 1969) has not reached the level of 
sophistication that benefits from the medium 
instructional approach. Or, they may not be achieved 
because the complexity of the topic requires different 
instructional representationsto fosterunderstandingfirst, 
then, retention and application. This study helps 
investigate these relationships by including these 
dimensions in the research design. 
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ABSTRACT 

The amount of business conducted electronically is, by some estimates, expected to grow to well over one trillion 
dollars by 2003. Along with the rapid growth comes a continual and constant redefining of terms, directions, and 
goals — both for the busin ess world and for academia . While Electronic Business (EB) and Electronic Commerce 
(EC) courses are being offered at many colleges and universities, few er schools sponsor entire degree programs in 
the area. As more businesses step into the electronic arena, the demand for "E-Smart" business managers will 
increase, along with the need for more focused degree programs. How have universities already responded to this 
ever-growing need? What new research programs and centers are being considered to handle the demand? In this 
paper, existing degree programs, concentrations, and research centers within the United States are examined. The 
research project presented herein is continuing and more EC program information will be presented at the lAIM 
conference in December 2000. 



INTRODUCTION 

While estimates of the amount of business conducted 
electronically vary, few would disagree that electronic 
business (EB) and electronic commerce (EC) have had 
and will continue to have a major impact on the global 
economy. As the impact grows, more managers will 
find it necessary to enhance their knowledge of 
electronic business (Khirallah 2000). Thus, it is not 
surprising that EB/EC courses are now being offered at 
colleges and universities throughout the world. The 
number of courses has risen significantly during the last 
few years, with five times more courses being offered in 
1999 than in 1997 (Sendall, 1999). 

While only a few years ago the single EC course was a 
rarity, now entire degree programs are being offered. 
Some universities generally the larger research 
institutions have started EC Research or Resource 
Centers. Furthermore, these research centers and degree 
programs are expanding rapidly. In 1995, Vanderbilt 
University started the first EC concentration with one 



student. The program now boasts "a dozen professors, 
nineteen courses and a center devoted to E-research 
(Lord 2000, p.62)." This year, U.S. News and World 
Report published a separate ranking for E-Commerce 
programs within its rankings of business school 
departments (September, 2000). 

Organizations, feeling the pressures to remain 
competitive in the EB world, are in turn pressuring 
colleges and unive rsities to develop programs in far less 
than the traditional time allotted to such curricular 
activities. Centers for EB research are appearing 
throughout academe, and more courses and degree 
programs are being implemented every semester. In this 
paper, the Research Centers and degree programs at 
more than one hundreduniversities throughout the world 
are examined to answer the following research 
questions: 

1. What universities have Electronic Commerce 
Research Centers/Institutes? 

Center name 
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School/college/department responsible for 
center 

Stated purpose of center 

2 , What degree programs are being offered? 

Graduate (MBA/MS) and undergraduate 

Program focus (technical/ma nagerial/ 
marketing/etc.) 

Tracks/concentrations/courses offered within 
the degree 

This research is ongoing. Internet searches for new EC 
centers, degree programs, and courses are performed 
monthly and new information is then added to the 
collection. Thus, the data presented in this paper will be 
updated at the lAIM conference in December 2000. 

METHODOLOGY 

The research project began with an Internet search for 
Electronic Commerce centers, degree programs, and 
course offerings at AACSB-accredited schools. The web 
site for each school was explored to discover to what 
extent the school had entered the EC arena. This 
research was naturally limited due to the nature of the 
World Wide W eb: schools that do not index orprovide 
relevant information on their web sites would not be 
included in the results. In some cases, schools were 
contacted to obtain further information. Some Internet 
searches yielded web sites that posted lists of Electronic 
Commerce programs. These web sites are listed in the 
references. 

During the course of the Internet searches, several email 
requests for information about EC programs were sent to 
the ISWorld listserv by MIS faculty in other institutions 
(Gupta 2000; Press 2000). The results of those requests 
were compiled and published on the list. These 
compilations were then used in this research and 
information about the programs added to the results 
herein. One request yielded an additional web site that 
included more information about EC programs 
(Warkentin, 2000). More recently, a search for 
international EC programs has been conducted. The 
results of that search is not included herein, but will be 
presented at the conference. 

RESEARCH FINDINGS 

The various Internet searches and listserv requests 
yielded information regarding a number of Electronic 
Commerce Research Centers, extensive graduate and 



undergraduate degree programs, certificate programs, 
and courses offered at the universities. In this section, a 
summary of the findings is presented. 

Electronic Commerce Research Centers 

There are a number of Electronic Commerce Research 
Centers located in university settings. While some of 
these centers are wellestablished(e.g. Vanderbilt's eLab 
was founded as Project 2000 in 1994), new centers are 
being created each year. Thus, compiling this list was 
truly an attempt to shoot at a fast moving target. In some 
cases, web sites did not explicitly indicate a research 
center, so it is probable that many more centers exist. 

For each center found, the name, web address, stated 
purpose and some details are presented in Table 1. 
Interestingly, only two centers clearly state that they are 
inter-disciplinary (UT Austin and Carnegie Mellon), 
although most of the centers involve more than one area 
within the business school. These centers are all linked 
to degree or certificate programs through the university; 
these programs are presented later in the paper. 

Electronic Commerce Degree Programs 

In the last few years, the number of degree programs in 
Electronic Commerce has skyrocketed. Schools now 
offer (at the undergraduate level) degrees in EC, EC 
concentrations within a general business. Management, 
MIS or marketing degree, and certificate programs in 
EC. At the graduate level, many schools offer an MBA 
concentration in EC as well as an MS in EC. Some 
schools also offer a graduate certificate in EC. In this 
section, more detail regarding these degree programs is 
provided. 

Ofthe 1 22 universities examined in thisresearch project, 
only four offer an undergraduate degree in Electronic 
Commerce. These schools are: University of Toledo, 
University of Scranton, Texas Christian University, and 
University of South Alabama. Three schools offer a 
concentration in EC within another major area. The 
major areas are Information Systems, General Business, 
and Marketing. No schools were found that offered an 
undergraduate certificate program, but two schools offer 
an EC minor. This info rmation is summ arized in T able 
2 . 

At the graduate level, there are many more degree 
programs. Fifteen schools offer an MS in Electronic 
Commerce. Six schools were found to offer an M S in 
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other areas with an EC concentration. Two schools 
offering the EC concentration have degree programs in 
Management; the degree program at the other four 
schools is in IS. The most popular degree program is the 
Electronic Commerce concentration within the MBA 
degree. Twenty-nine schools were found that offer this 
option. As might be expected, the MBA programs 
appear to focus on either thetechnical or the managerial 
aspects of EC; however, it was not always possible to 
discern this from the web sites. This information is 
summarized in Tables 3, 4 and 5. 



Only four schools were found to offer graduate 
certificates in EC, and these programs were quite 
diverse. At Georgia Institute of technology, for 
example, students can concentrate in one of four areas: 
electronic management, networking, interface design or 
multimedia. At Southern M ethodi st University, there are 
three distinct tracks offered: The EC Professional track, 
the Application Developer track, and the Security 
Specialist track. 



TABLE 1 

UNIVERSITY ELECTRONIC COMMERCE RESEARCH CENTERS (U.S. SCHOOLS) 



University 


Location 


Name 


Web Address 


Purpose (as stated on the web 
site) 


Brigham Young 
University 


Business 


Rollins Center for 
eBusiness@BYU 


http://marriottschoo 
1. byu.edu/ebusiness 
/about/index. htm 


...fosters mutually beneficial 
relationships among faculty, 
students, and industry. In 
addition, the Center supports 
multidisciplinary research 
pertaining to innovative 
Information systems strategies, 
eBusiness, high-tech research, 
information technology 
application, and consulting. 


Carnegie Mellon 
University 


Business, 
Public 
Policy and 
Computer 
Science 


Institute for E- 
Commerce 


http://w ww.e com .c 
mu.edu/ 


...to serve as a focal point for 
Electronic Commerce activities 
at Carnegie Mellon University, 
located in Pittsburgh, PA, to 
offer degree programs and 
executive education and to 
conduct research. 


Creighton 

University 


Business 


Joe Ricketts 
Center in 
Electronic 
Commerce and 
Database 
Marketing 


http://ec omm erce.c 
reighton.edu/ 


... to be a focal point fore- 
commerce education and a 
resource base for the business 
community in e-commerce and 
database marketing. The center 
partners with local and national 
businesses to share knowledge 
about e-commerce with 
educators, students and other 
businesses. The center will 
sponsor electronic business 
education programs and research 
as well as organize conferences 
on e-commerce and database 
marketing. 


Emory 

University 


Business 


Center for 
Electronic 


http://www.cc.emo 

ry.edu/BUSINESS/ 


...a means of coordinating 
research and practice 
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Commerce 


welcome.html 


in the important areas associated 
with future forms of commerce. 
The center is organized to 
conduct research, facilitate 
informative and collaborative 
meetings, deliver reports and 
presentations, facilitate industry 
trade group initiatives, and 
promote industry revitalization 
and preparation for 21 st century 
business practice. 


Georgia Institute 
of Technology 


Business 


iXL Center of 

Electronic 

Commerce 


http://www.mgt.gat 

ech.edu/centers/cen 

ters_ixl.shtml 


...to sponsor state-of-the-art 
research and education in the 
emerging field of electronic 
commerce. The Center 
provides recommendations on 
how best to solve e-commerce 
challenges and problems, using 
real-world businesses as subjects 
for research projects and giving 
graduate students hands-on 
experi ence. 

The research and teaching 
activities of the Center focus on 
six key areas: online marketing, 
strategic uses of business-to- 
business inter-organizational 
systems, information security, 
information technology-enabled 
entrepreneurship, business 
intelligence, and business 
models for Internet-based 
ventures. 


Georgia State 
University 


Business 


eCommerce 

Institute 


http://www.eci.gsu. 

edu/ 


To discover, assimilate, facilitate 
and disseminate knowledge 
regarding all facets of e- 
commerce (business-to-business, 
business-to-consumer) through 
programs of education, research, 
incubation and outreach; in so 
doing, to position the Institute 
and its faculty, staff, students 
and partners as trusted sources 
of competence and knowledge in 
this area, to better promote and 
effect change in organizations 
and the markets they serve. 
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University 


Location 


Name 


Web Address 


Purpose (as stated on the web 
site) 




Business 


The Center for 
Digital Commerce 
(research center 
within the 
eCommerce 
Institute) 


http://www.eci.gsu. 

edu/research.htm 


to promote cross-disciplinary 
collaboration among the many 
areas impacted by e-commerce. 
The Center seeks to be a major 
resource for Georgia in 
attracting and retaining 
"industries of the mind." 


La Salle 
University 




Electronic 

Commerce 

Institute 


http://alpha.lasalle. 

edu/academ/ecom 

merce 


...promote the field of e- 
commerce. The Institute will 
utilize faculty and business 
professional expertise so e- 
commerce learning may be 
applied to the entire student 
population. The Institute will 
apply real world applications 
to develop an e-commerce 
knowledge base. 


MIT 


Business 


Center for 
eBusiness® MIT 


http://ebusiness.mit 

.edu/ 


...to provide leadership for 
faculty, students, and industry 
interested in Internet-enabled 
business. 

Our mission is to be the leading 
academic source of innovation 
in management theory and 
practice for eBusiness. 


Stanford 

University 


Business 


Center for 
Electronic 
Business and 
Commerce 
(CEBC) 


http://www.gsb.sta 

nford.edu/cebc/ 


... the development of new 
research and courses to 
understand the impact of 
information and communication 
technologies on firms, industries 
and 

markets. 

... to leverage existing 
knowledge creation and 
curriculum development efforts 
and start new ones. 


Temple 

University 


Business 


Irwin L. Gross 

eBusiness 

Institute 


http://www.sbm .te 
mple.edu/ebi/ 


Laboratory. Classroom. Forum 
for dialogue. Think tank. Project 
applications group. Business 
incubator. Virtual library and 
resource center 

eBI leverages The Fox School’s 
research expertise, 
educational resources, global 
presence and entrepreneurial 
spirit to prepare business leaders 
and help them develop 
strategies and models for the 
digital economy 
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University 


Location 


Name 


Web Address 


Purpose (as stated on the web site) 


University of 
Texas Austin 


Business, 

Computer 

Science, 

Law, 

Economics, 
Communic 
ation and 
Library 
Science 


Center for 
Research in 
Electronic 
Commerce 


http://cnec.bus 

.utexas.edu/ 


...seek to provide an efficient and 
effective framework for global 
electronic commerce and the digital 
economy through an integrated 
research agenda that focuses on 
correlative effects among network 
infrastructure, products, processes, 
payment systems and policies, using 
markets and economic analysis as the 
central unifying theme. 

The Center’s vision is to assure that 
electronic commerce processes and 
applications achieve their efficient 
outcomes promised for the digital 
age. 


Vande rbilt 
University 


Business 


eLab 

Electronic 

Commerce 

Research 

Laboratory 


http://ecomme 
rce. Vanderbilt, 
edu/ 


...a corporate sponsored research 
effort devoted to the scholarly and 
rigorous investigation of the 
marketing implications of 
commercializing hypermedia 
computer-mediated environments 
(CMEs) like the World Wide Web 
and other emerging electronic 
environments. 

Elab research objectives are to enrich 
and stimulate the knowledge base on 
the role of marketing in new media 
environments, provide a principal 
point for the discussion and exchange 
of these ideas, and impact business 
practice in this emerging area. 



TABLE 2 

UNDERGRADUATE PROGRAMS IN ELECTRONIC COMMERCE 



Program 


School 


Degree 


Undergraduate Major 


Texas Christian University 


BBA in Electronic Business 


University of Scranton 




University of South Alabama 


University of Toledo 


BBA in Electronic Commerce 


EC Concentration 


Old Dominion University 


BSBA Information Systems 


BSBA Marketing 


Rutgers 


BS Management 


Utah State University 


BS in BIS 


Undergraduate Minor 


University of Toledo 




University of West Florida 
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TABLE 3 

GRADUATE PROGRAMS IN ELECTRONIC COMMERCE: MS DEGREE 



University 


Degree Name 


Example Courses Offered 


Arizona State 


MSIM in Electronic 
Commerce 


Technical Foundations of EC 
Business Database Concepts 
Distributed IS 


Barry 


MS in Electronic Commerce 


Not available 


Carnegie 

Mellon 


MS in Electronic Commerce 


EC Management 
EC Technology 
Core java in EC 


Capitol College 


MS in Electronic Commerce 
Management 


Managing the Global Virtual Organization with Technology 
Leadership, Motivation and Obtaining Results at a Distance 
Business Technologies 


Claremont 

Graduate 

School 


MS in Electronic Commerce 


Database Concepts 
Electronic markets 
Electronic Economies 


Creighton 


MS in Electronic Commerce 


Introduction to EC 

Development Technology for the Web 
Special Topics in EC 


DePaul 


MS in Electronic Commerce 


Usability Issues 
Advanced WEB IS 
Extranet Systems 


Golden Gate 


MS in Electronic Commerce 


Electronic Commerce Industry and Technology 
Electronic Commerce Systems and Technologies 
Professional Practice Project in Electronic Commerce 


Illinois Institute 
of Technology 


MS in Electronic Commerce 


E-commerce Organization Design 
Customer Relationship Management 
E-Commerce Security & Payment Systems 


National 

University 


MS in Electronic Commerce 


Principles of Electronic Commerce 
Embarking on Electronic Commerce 
Global Network Marketing and Advertising 


Northwestern 


MS in Electronic Commerce 


Fundamentals of Technology and E-Commerce 
Intellectual Property Strategies for Managers of E-Commerce 
Internet Business Models and Technology 


Notre Dame 
California 


MS in Electronic Business 
Management 


Foundations ofthe Digital Economy 
Fundamentals of Internet Architecture 
Marketing & the Internet 


Rice 


MBS in Electronic Commerce 


Strategic Services Management 
Internet Infrastructure 
Electronic Commerce 


San Francisco 
State 


MSBA in Electronic 
Commerce 


Seminar in ED Site Design and Management 
Seminar in Distributed Data Processing 
Internet Marketing 


Temple 


MS in Electronic Commerce 


Information Architecture and networking 
IS Management & Process 
Database Analysis and Design 


University of 
San Diego 


MS in Electronic Commerce 


Introduction to EC 

Web Site Design and Management 

Ethics and Law for Online Organizations 


Wisconsin - 
Milwaukee 


MS in Electronic Business 


Introduction to Electronic Business 
Development of EB applications 
Analysis and Design of E-systems 
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TABLE 4 

GRADUATE PROGRAMS IN ELECTRONIC COMMERCE; MS WITH EC CONCENTRATION 



School 


Degree Area 


Example Courses 


Denver 


Information Technology 


Systems Analysis & Design 
Introduction to EC 
DBM Systems 


Georgia State University 


Information Systems 


Electronic Marketing 
Economies of Digital Business 
EC Infrastructure 


Loyola University - Chicago 


Information Management 


Website Planning & Development 
Internet Marketing 
Electronic Commerce 


New Jersey Institute of Technology 


Management 


Internet for Managers 
Internet Marketing Strategy 


North Carolina State 


Management 


Managing the Digital Enterprise 
EC Technology 
Networking for EC 


Purdue 


Management 


EC & Supply Chain Management 
E People 

Competition in EC 


Rochester Institute of Technology 


Information Technology 


IT & Organizational Process 
Inter-Enterprise Computing 
IT & Strategic Opportunity 



TABLE 5 

GRADUATE PROGRAMS IN ELECTRONIC COMMERCE: MBA WITH EC CONCENTRATION 

Schools 



Arizona State 
Brigham Young 
Denver 
Drexel 
Emory 

Florida Atlantic 

Georgia State University 

Illinois Institute of Technology 

Loyola - Chicago 

Maryland 

MIT 

National U niversity 

New Jersey Institute of Technology 

Notre Dame 



Old Dominion 

San Francisco State 

Santa Clara State 

Southern Illinois - Edwardsville 

Temple 

University of North Carolina - Chapel Hill 

University of Pennsy Ivania 

University of Rochester 

University of Texas - Austin 

University of Washington 

Vanderbilt 

Virginia Tech 

Wake Forest 
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In some cases, the web site did not offer enough 
information to determine the exact degree offering. 
Northwestern University graduate school, for example, 
offers a Technology and E -Com merce " program " within 
which some nineteen courses (across the technology, 
marketing, management and operations areas) are 
offered, but there are no specifics regarding the degree 
issued upon completion. 

EC Course Offerings 

A large number of schools that do not offer full degree 
programs in EC now offer Electronic Commerce 
courses. While the majority of schools (48) still offer 
only one EC course, that is beginning to change. 
Thirteen schools offer two courses in EC, and three 
schools offer three courses. One school offers four 
courses in EC. Of the schools offering only one course, 
thirty-five of them offered the course in the IS area. 
Other areas represented included: Marketing (2), 
Management (3), Accounting (1), and General Business 
(5). Two schools listed their courses with an MBA 
designation only. Twenty-two of the schoolsofferedthe 
course at the graduate level, while twenty-one schools 
had undergraduate co urses. Five schools did not specify 
the level of the course. 

For schools offering two courses, there was more 
diversity among the areas represented. Three schools 
split the courses between IS and Marketing, two schools 
offered both courses under Management, one school 
offered the courses in Accounting and Business, and five 
schools offered both courses in IS. Four of the schools 
offered only graduate classes; four offered both classes 
at the undergraduate level and four were split, offering 
one course at each level. 

Only three schools offered three courses without 
specifyinga concentrationor degree program. For these 
schools, the course offerings were at the graduate level 
and were offered by the IS department. The school that 
offered four courses without specifying a concentration 
or degree program in EC offered them all at the graduate 
level with an MBA designation. From the titles of the 
courses, however, it is apparent thatthey spanned several 
business disciplines (Technology, Strategy and 
Marketing). 



SUMMARY 

As Electronic Comm erce interests in the busine ss world 
continue to grow, it is reasonable to expect that 
universities will seek to expand their programs and 
course offerings. As these universities look to develop 
electronic business or electronic commerce 
concentrations or degree programs, they can learn from 
programs that already exist. This is evidenced by the 
requests to the ISWorld listserv for course and program 
information regarding E-commerce. 

In many respects, compiling a list of E-commerce 
programs is a daunting task, as the number ofprograms 
seems to grow rapidly. New research centers and 
initiatives, new course offerings and degree programs 
and new certificate programs become available every 
month. Given this rapid program development, we may 
be approach ing a time when universities need to consider 
if offering an EC program is a viable option or overkill. 
In light of the critical shortage of IS faculty at many 
institutions, is it wise to begin a new program that 
further taxes the staff? 

It is hoped that this paper provides some insight to those 
faculty and institutions seeking to develop EB/EC 
programs and forthose professors teaching such courses 
at either the undeigraduate or graduate level. 
Furthermore, those responsible for IT curriculum and 
schools involved in AACSB accreditation or re- 
accreditation efforts may also find the research helpful. 
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EXPERIENTIAL LEARNING 
IN THE INTRODUCTORY MIS CLASS: 
INTERVIEWS WITH IT PROFESSIONALS 
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ABSTRACT 



Students enter the introductory MIS class having heard all the hype regarding the IT profession, but often knowing 
little of the realities. In an effort to have general business students gain a deeper understanding of the IT profession 
and the IT professional, the authors developed an interview exercise that serves many purposes. Students receive 
instructionon conducting interviews and the researchproject in which they are participating is described. Then each 
student selects an appropriate interview candidate and researches the candidate's organization. Each student 
conducts an in-depth interview, including both a structured and an unstructured interview. Each student submits a 
paper describing the organization and reporting on interview results. Finally, each student makes a formal 
presentation to the class. This article details the methodology for this pedagogical exercise and reports results 
obtained after over one hundred interviews. Benefits achieved are substantial from a pedagogical, research, and 



community-relations perspective. 

INTRODUCTION 

The introductory Management Information Systems 
(MIS) class at the undergraduate level in most 
universities represents the only opportunity most 
business students have to learn about the Information 
Systems (IS) or Inform ation Technology (IT) profession. 
The topics covered in this course vary greatly among 
schools. (See O’Hara and Stephens, 1999a and 1999b 
and Stephens and O'Hara, 1998 for more details 
regarding course topics and requirem ents), Wh ile there 
is typically some exposure to the real world, it is usually 
limited to exploring case studies, performing web-based 
research, or listening to guest speakers in class. 

One way to allow students to explore the profession is 
for them to conduct field interviews with IS 
professionals. This interview activity serves anumber of 



useful purposes. First, it provides the student with a 
unique opportunity to develop their research skills by 
conducting a formal interview with the professional. It 
also allows students to interact with someone in the 
IS/IT field. This is very often the first formal interaction 
students have with these professionals. This experiential 
learning exercise has proved very valuable to the 
students. Second, since the interview is preceded by 
hands-on training in conducting an interview, students 
gain valuable skills they can use throughout their 
business careers. Before conducting the interview, 
students must explore the company they plan to visit — 
either doing web-based (preferred) or library-based 
research. Students then write a synopsis of the company. 
After conducting the interview, students present their 
findings to the class, share the insights they gained from 
the process, andsubmit all written documentation to the 
instructor. 
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In this paper, the authors firstprovide some background 
on experiential learning in the IS class. They then offer 
details concerning the interview assignm ent, and how the 
assignment helps studentshone a variety of skills. While 
the purpose of this paper is not to explore the results of 
the interviews, the major insightsand lessons learned by 
students are also shared. 

EXPERIENTIAL LEARNING 
IN THE IS CLASS 

Experiential learning may be simply defined as active 
participation in the learning process. In general, 
experientiallearning(oreducation) involves "immersing 
students in an activity (ideally closely related to course 
material) and then asking for their reflection on the 
experience" (Clements, 1995, p. 116). The value of 
experiential learning has been known for some time. As 
early as 1916, Dewey stated: 

In schools, those under instruction are too 
customarily looked upon as acquiring 
knowledge as theoretical spectators... the very 
word pupil has almost come to mean one who is 
engaged not in having fmitful experiences but in 
absorbing knowledge directly (Dewey, 1916, 
p.l64). 

Carl Rogers, the noted psychologist, defined two types 
of learning: meaningless and significant. Meaningless 
learning involved memorization and cognitive process; 
significant learning was directed to the specific needs 
and wants of the learner. For Rogers, experience was 
paramount: 

The touchstone of validity is my own 
experience. No other person’s ideas, and none of 
my own ideas, are as authoritative as my 
experience (Rogers, 1995, p. 63). 

To Rogers, theteacher's role was to facilitate significant 
learning through experience. 

Experiential learning currently plays a significant role in 
higher education. Programs exist in Arts and 
Humanities, the Social Sciences, and the Professional 
and Technical fields (Cantor, 1997). These programs 
take many forms, including role-playing, community- 
based work programs, and internships. Still, experiential 
learning does not need to be so formalized. It can occur 
more simply when faculty provide the means for 
students to gain more relevance fi“om their educational 



experiences, something students are demanding (Matson 
and Matson, 1995). 

Experiential learning has been popular in MIS classes as 
well. MIS faculty have long recognized the need forand 
value of student exposure to the workplace (Dick and 
Jones, 1995), and often include a real-world component. 
In the System s Analysis and Design class, for example, 
students often interview each other to learn various 
information gathering techniques and design systems for 
small businesses in the community. Another class in 
which a real-world project is often undertaken is the 
Database class. Inclusion of a real-world project in the 
introductory class is mu ch less common. Students in this 
class cannot typically develop an information system, 
nor is there is knowledge extensive enough to complete 
an internship. F or these students, an altern ative to a full- 
scale project is an interview with an IS professional. 

THE INTERVIEW ASSIGNMENT 

The specifics ofthe interview assignment have changed 
since it was first used in the MIS class. Initially, 
students simply found a likely candidate, conducted an 
unstructured interview and reported their results to the 
instructor in writing. During the last two years, the 
assignment has evolved into a multi-faceted, multi- 
purpose project. Results of the interviews conducted by 
students are being studied by the faculty involved and 
will be reported on in another venue. 

The first aspect of the project is developing interview 
skills. Some students, especially the traditional ones, 
have never taken part in an interview — on either side of 
the desk. Most who have participatedin interviews have 
been the interviewees, not the interviewer, so this is a 
new role for them. Thus, at least one full class (75 
minutes) is devoted to developing interview skills. 
Topics such as courtesy, staying focused on the 
interview questions, providing verbal and non-verbal 
feedback, and closing the interview are discussed. The 
final exercise of the interview skills class is a hands-on 
one. Students practice the first few moments of the 
interview — often the most awkward — with the 
instructor. They "enter" the interviewee’s office, shake 
hands, make small talk and get started on the interview. 

I 

Next, the interview form itself is distributed to the 
students electronically. The form consists of two parts: 
The Structured and Unstructured Questions. The 
Structuredlnterview section has three sub-sections: The 
IT Profession, Job Satisfaction, and Work Environment 



Proceedings of the 15'*’ Annual Conference of the International Academy for Information Management 



71 



(See Appen dix A). These questions were developed and 
pre-testedovertwo semesters. Students are told to begin 
thinking about what questions they might want to ask 
during their interview, and to continue to formulate 
questions as they determine their interviewee and 
conduct their company research. As interviews are 
conducted and presented to the class, students gather 
ideas for the unstructured questions they will ask. 

Students then find an IS person to interview . Oddly, this 
has never proved problematic for them — there always 
seems to be enough IS people available and willing to 
participate. Students are free to interview people who 
work in any IS capacity — from help desk to CIO. This 
allows for a broader base of information to be shared 
with the class. Interv iews m ay be tape d; this is mutua lly 
determined by the student and interviewee. Students are 
encouraged to keep the interview time under one hour to 
accommodate busy IS professionals. 

Students must follow strict guidelines when setting up 
the interview. They must get the interviewee's 
permission to be identified to the instructor (identities 
can be kept from the students in the class). Participants 
must also agree to take part in a possible follow-up study 
from the university. Students must call to confirm the 
interview two working days before it is scheduled. 
When the interview is completed, students must send the 
IS professional a copy of the interview for review, and 
obtain a signed statement from the participant that the 
information is factualand accurately presented. Finally, 
students must follow-up the interview with a Thank You 
note, a copy of which must be submitted to the 
instructor. 

Once the interviewee is selected, the student submits the 
person's name, title andjob description to the instructor 
for approval. Only rarely are interviewees deemed 
unacceptable. Reasons for such have included duplicate 
interviews (i.e., the previous semester) and an 
overabundance of interviewees in the same areas (e.g., 
programmers). Discussions of the interviews are 
scheduled to coincide as much as possible with course 
topics. Thus, interviews with network engineers and 
administrators are presented during the 
Telecommunications chapter; while Database 
Administrator interviews are discussed while covering 
the database chapter. 

Once the interview subjects have been approved, 
students begin their company research. Using the World 
Wide Web, they explore the company history — both 



financially and culturally if possible. They may also 
obtain more information regarding the job title of the 
interviewee to ask informed IS-related questions. Web 
site evaluation is another required assignment in the 
course, and if the company has a public web site, 
students may choose to evaluate the site using criteria 
developed in class. Once their research is complete, 
students must type a brief (one page is fine) synopsis of 
the company and include a works cited page. 

Students present their interview results in groups. The 
day of their presentation, they must have the interview 
form completed, a PowerPoint presentation of the 
interview details, and all other supportin g 
documentation. The scheduled presenters gather in the 
front of the room and each presents for about seven 
minutes. Typically, there are no more than five students 
in each group, so the first phase takes about a half hour. 
After all the students have presented, there is an open 
forum for discussio n. The form al presentatio n portion is 
used to present thefactual parts of the interview (i.e., the 
structured section). The open forum begins with 
students sharing the information they gleaned from the 
unstructuredportion. These forums typically generate an 
active class discussion. 

Student response to the interview exercise has been 
overwhelmingly positive. Many students mention the 
assignment specifically on their end of semester 
evaluation forms as being the one from which they 
learned the most. As was demonstrated above, the 
interview assignment is extremely valuable to the 
student for a number of reasons: 

It involves students in a real world situation 
It provides students with useful interviewing skills 
It allows students to participate in a research project 
It provides students with an opportunity to analyze 
a web site 

It affords students an opportunity to hone verbal and 
written com munication skills 
It requires students to use computer skills learned 
previously (e.g. PowerPoint) 

It strengthens the university relationship with the 
business community 

One unexpected benefit of the interview has been in job 
placement services. One student interviewed three 
professionals in different IS areas. He felt he had 
learned so much about possible employment from the 
first interview that he would conduct more. By doing so, 
he was able to glean great insightinto the various career 
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paths available to him. Several students have been 
offeredjobs by thepeople they interviewed; some others 
have taken semester-long internships working with the 
people they met during the assignment. 

The interview exercise is also quite valuable to the 
instructor, the MIS department, and even the business 
school. The interviews generate a vast amount of data 
that can be analyzed and reported upon in later research 
projects. Issues such as work environment, turnover 
rates, and job satisfaction have all been covered in 
interviews. Thus, faculty can benefit from the research 
papers the interviews produce. The interviews have also 
provided information as to the issues the professional see 
as importantforbusinessstudents. Thus, the interviews 
tap into an information source that departments and 
schools can use as a first source for decisions regarding 
course and curriculum issues. Further, because the 
students are out in the community, the interview process 
can raise the visibility of the IS program and establish a 
link to the business community. 

MAJOR INSIGHTS AND 
LESSONS LEARNED BY STUDENTS 

While the data gathered in these interviews hasprovided 
the authors with a wealth of information for other 
research efforts, the primary purpose of this paper was 
not to share the results of the interviews. Still, the 
students' insights and lessons learned are germane to a 
discussion of the value of the exercise. In this section, 
the authors present some of those lessons. 

Students were most surprised by the consistent level of 
job satisfaction of the IS employees versus the number 
of hours worked. In interview after interview, subjects 
reported that they were satisfied with their jobs, despite 
long working hours. While the average work week for 
the IT professionals was over 50 hours, most reported 
being at least somewhat satisfied with their jobs. 
Moreover, all but eleven of the interview subjects were 
requiredto spend time on-call, typically one week every 
six weeks. 

Another surprising aspect of the interviews was the 
number of subjects who told the students not to make 
money their first priority in finding a job. While the 
goals and objectives (other than financial) varied greatly 
among the interviewees, some of the areas mentioned 
included: finding a firm that will offer a broad range of 
experiences, working for a company that allows you to 



stay current with respect to IT skills, and doing 
something you enjoy. 

One interview question specifically targeted the IS 
majors. These students were suiprised to learn some of 
the advice the interviewees had for them: learn a broad 
range of skills, knowing technology is never enough, and 
focus on the training you will receive rather than the 
salary. Among the skills deemed important to IT 
professionals were: communications (both verbal and 
written), strong business background, project 
management, general management skills, and team work 
skills. 

CONCLUSIONS 

The interview exercise serves a multitude of purposes in 
the introductory MIS class. It offers the students a 
chance to improve interviewing, writing, and 
presentation skills. Further, students active ly participate 
in a research project. It provides the instructor with 
valuable research data, and it establishes a link between 
the business school and the IS community. Insights 
gained from the interviews can be used to generate class 
discussion and assist students in their search for the 
appropriate career. 
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APPENDIX A 
THE INTERVIEW 

CISM INTERVIEW WITH IT PROFESSIONAL 



Name: Company: 

Job Title: Mailing address: 

E-Ma il : 

Phone: 



PART I: STRUCTURED INTERVIEW 



Section 1: IT Profession 

1. How many years have you been in an information systems or information technology job?. 

2. How many IT job changes have you experienced with your present firm? 

3. How many company changes have you experienced while in an IT job? 

4. How many geographic relocations have you experienced while in an IT job? 

5. What were the major reasons for job changes? (within same firm)? 



6. For company changes? 
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7. 



What are the five key (general business) knowledge areas for today's information systems majors? 



1. 

2 . 

3. ^ 

4. 

5. 

8. What are the 5 key (technology) skill sets for today's information systems majors? 

1. ^ 

2 . 

3. 

4. 

5. 

9. What do you believe to be the 5 key issues facing IT professionals today? 

1 . 

2 . 

3. 

4. 

5. 

10. What are the three major ways you keep your IT knowledge and skills current? 

1 . 

2 . 

3. 

1 1 . Which level of management support for keeping current best describes your situation? 

1-none 2-very little 3-some but inadequate 4-adequate 5-good 6- very good 7- excellent 



12. What advice would you give an information systems major who has just graduated and is seeking a job in the 
field? 



Section 2: Job Satisfaction 

1. What is the average number of hours you work per week? 

2. What is the range of hours per week B a maximum and minimum ? 
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3. Are you on call? If so, what is your on call schedule? 



4. Which description best fits your level of job satisfaction: 

1- very dissatisfied 2- dissatisfied 3- somewhat dissatisfied 4- neutral 

5-somewhat satisfied 6- satisfied 7- very satisfied 



Section 3: Work Environment 

(student observation if interview done face-to-fece; ask if telephone interview is conducted) 

1 . Office or cubicle? 

2. If office, high wall or low wall? 

3. Approximate size? 

4. Which phrase best describes the sound level? 

1-loud and distracting 2-frequently distracting 3-occasionally distracting 
4-not noticeable 5-other noise does not penetrate work area 

5. Are there headsets for listening to music? 

6. Is there a window? If so, describe the view (natural scenery, industrial view, pavement, etc.). 



7. If so, does the window open? 

8. Is there natural light? 

9. Describe the lighting fixtures (glare/non-glare; fluorescent and/or area lighting, etc.). 



10. What amenities are present? 

guest chairs 

conference table 

decorative items (wall hangings, diplomas, etc) 

personal souvenirs and photographs (brief description) 

others (please describe) 

1 1. (For students to complete) Would you want to work in this environment? Why or why not? 
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Part n: UNSTRUCTURED INTERVIEW 



For this part of the interview, you m ay ask the IT professional any questions you want. They should be open-ended 
questions so that you obtain rich information rather than simple "yes or no" answers. 

Try to ask questions that flow from information you may have learned in the first part of the interview. 
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Anderson (1982, 1993) proposed that individuals progress from possessing declarative knowledge to possessing procedural 
knowledge as they acquire cognitive skills. Since cognitive skill acquisition is an important part of an information ^sterns (IS) 
education, it is proposed that evaluating IS students on both declarative and procedural knowledge within skill domains will 
provide several educational benefits to students and educators. This initial study examined whether an objective format (i.e., 
multiple-choice) examination on data m odeling cou Id discriminate between declarative and proced ural knowledge in IS students. 
The subjects for the study were IS students in two different courses. One of those courses was a prerequ isite for the other course, 
so students had had different levels of experience with data m odeling. The resu Its indicated that there was a significant difference 
in the performance of the subjects in both classes between the questions testing declarative knowledge and those testing 
procedural knowledge. Also, while there was no significant difference between the two classes in their performance on 
declarative-type questions, there was a significant difference in their performance on procedural-type questions. These results 
provide initial evidence thatan objective format examination can discriminate between the two types of knowledge, providing 
educators and students with insight into th e students ' acquisition of the cognitive skills being taugh t. 



INTRODUCTION 

Information Systems (IS) professionals are required to 
display a wide range of skills (Cheney, Hale and Kaspar, 
1990; Dwight, 1993; Lee, Trauth and Farwell, 1995, 
Leitheiser, 1992; Misic, 1996; Prabhaker, Litecky and 
Arnett, 1996), and most of these skills tend to be 
cognitivein nature. As educators ofIS professionals, we 



need to be able to develop each of these skills in our 
students through instruction and evaluation. Cognitive 
skill acquisition theory was consulted to provide 
guidance in the evaluation of IS students’ cognitive 
skills. 

Cognitive skill acquisition theory (Anderson, 1982, 
1 993) proposes that cognitive skills are acquired through 
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a process of moving from declarative knowledge to 
procedural knowledge. As a student begins to acquire a 
new cognitive ski 11, declarative knowledge intheform of 
facts and examples is stored in memory. Examples 
consist of problems specific to the skill domain, as well 
as the steps taken to solve those problems (VanLehn, 
1996). When students attempt to solve problems on 
their own, they search for an analogous example. They 
then use that example’s solution to arrive at their own 
solution for the problem on which they are currently 
working. This initial mode of problem solving has been 
shown to be mo re tedious a nd to require more time than 
the problem solving demonstrated by experts (Anderson, 
1993). 

As problems are solved, students are able to com pile this 
initialdeclarativeknowledge into procedural knowledge. 
Procedural knowledge consists of knowing how to 
perform a certain skill. Because it consists of 
psychomotor or cognitive actions, it is difficult to 
describeprocedural knowledge in verbal or written form. 
Verbal or written descriptions should notbe equated to 
the procedural knowledge they describe. Reading a 
description of procedural knowledge does not provide 
one with that procedural knowledge. Either the person 
reading the description already possessesthe procedural 
knowledge and knows how to apply that which has been 
read, or the person must still acquire the knowledge 
through experience in performing the skill. 

Evaluations that help to establish an IS student’s 
possession of declarative and procedural knowledge 
could be used to prescribe future training and classroom 
exercises. Any deficiencies in declarative knowledge 
could be addressed by further instruction in the 
conceptual basics of the skill domain. Deficiencies in 
procedural knowledge would indicate a need for further 
experience in using the skill. Knowing which 
production rules are deficient would aid in providing 
activities that would help to develop those specific 
production rules. 

The purpose of this study is to examine whether 
procedural knowledge in IS students can be evaluated 
with an objective examination format. A multiple- 
choice format was chosen for the instrument because 
multiple-choice tests have been used in the past for 
cognitive testing(Haladyna, 1999), and because of their 
objective nature. 



RESEARCH QUESTION 

Examinations that are based on the dichotomy between 
declarativeand procedural knowledge have the potential 
for several benefits for both instructors and students in 
IS. These benefits include a betterunderstanding of the 
student’s progress in acquiring the skill and an ability to 
objectively evaluate the student’s progress. This initial 
study therefore asks the following research question: Is 
an examination in an objective format (e.g., multiple 
choice) capable of differentiating between declarative 
and procedural knowledge in IS students? 

Research has demonstrated that novices tend to possess 
declarative knowledge and lack procedural knowledge 
(Glaser, 1990), while experts possess procedural 
knowledge, as well as declarativeknowledge (Glaser and 
Bassok, 1989). Subjects were taken from two 
undergraduate classes thatteach data modeling: Systems 
Analysis and Design (SAAD) and Database Design and 
Administration(DDAA). There should be no significant 
difference between the two classes in terms of 
declarative knowledge because each of the subjects had 
received instruction in data modeling when they 
completed the examination. On the other hand, the more 
advanced class (DDAA), should outperform the less 
advanced class (SAAD) on items that examine 
procedural knowledge, since the students in the more 
advanced class will have had more opportunities to 
develop procedural knowledge. 

Anderson’s Adaptive Character of Though-Rational 
(ACT-R) theory (1982, 1993) provided the basis for the 
three hypothesis tests given in Table 1 . is the mean 
percent correct on the declarative questions, is the 
mean percent correct on the procedural questions, 
M-dccddaa is mean percent correct on the declarative 
questions for the Database Design and Administration 
course, Pdccsaad is mean percent correct on the 
declarative questions for the Systems Analysis and 
Design course, PprocooAA is the mean percent correct on 
the procedural questions for the Database Design and 
Administration course, andpp^^ps^^^is the mean percent 
correct on the procedural questions for the Systems 
Analysis and Design course. The first hypothesis test 
compares the overall performance on the questions 
testing declarative knowledge to the questions testing 
procedural knowledge. The second andthird hypothesis 
tests compare the performance between the two classes 
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on either the declarative questions (the second 
hypothesis test) or the procedural questions (the third 
hypothesis test). It is expected that the first and third 
hypotheses will have significant results, while there will 
be no significant results with the second hypothesis. 



TABLE 1 

HYPOTHESIS TESTS 



Test 


Expected Results 


^lo* M'Dcc ftproc 

I^la* I^Dec ^ Itproc 


Significant 






^20* M'DccDDAA “ M^DccSAAD 
^5^: ItnecDHAA ^ M^DecSAAD 


Not significant 






^30* M'ProcDDAA ” M'ProcSAAD 

M-ProcDPAA ^ M-ProcSAAP 


Significant 



METHODOLOGY 

Development of the instrument began by surveying 
^subject matter experts to determine the minimum level of 
knowledge and skills needed by a systems analyst to be 
able to produce data models on the job. To develop the 
survey, standard textbooks in the areas of systems 
analysis and design (Fertuck, 1 995; Hoffer, George and 
Valacich, 1999; Kendall and Kendall, 1 999; Whitten and 
Bentley, 1998) and database design (McFadden and 
Hoffer, 1991; Sanders, 1995) were consulted. Defini- 
tions, concepts and proce sses for data modelin g given in 
each of these textbooks were listed. 

The list of 81 items obtained from the text books 
(Appendix A) was then given to nine subject matter 
experts and they were asked to consider the minimum 
level of knowledge and skills an employee in an entry- 
level data modeling job would need to complete three 
common tasks in data modeling. The three tasks were 
drawing entity relationship diagrams, completing a 
repository for those diagrams, and using a computer- 
aided software engineering (CASE) tool to draw the 
diagrams and complete the repository. The subject 
matter experts were asked to rate each item in the list on 
a 5-point Likert scale (where 1 was “Very low” and 5 
was “Very high”) based on the following question in 
relation to those three tasks. 



How importan t is it that the employee described 
above possesses this knowledge or skill when 
working on the above three tasks? 

Seven of the nine subject matter experts returned 
completedsurveys. The results are pro vided in Appendix 
B. Items with an overall median importance of less than 
3.0 (the midp oint ofthe scale) were dropped from further 
consideration. Items with a median importance of 
between 3.0 and 4.0 were carefully considered to 
determine whether their elimination would adversely 
affect the coherence of the training. Items that were 
dropped include those that pertained to logical and 
physical models; enterprise data models; database 
models (i.e., hierarchical, network, and relational); 
supertypes and subtypes; normal forms other than first, 
second, or third; and business rules. Also, the subject 
matter experts identified two additional itemsthat were 
added to the list (the last two items listed in Appendix 
A). 

Based on the resulting list, the instrument was 
developed. There are two types of questions on the 
instrument: those that test declarative knowledge and 
those that test procedural knowledge. The declarative 
knowledge questions were written based on standard 
definitions for constructs in data modeling and reasons 
for the importance of these constructs. 

The procedural knowledge questions required writing 
questions based on the production rules behind 
completing specific subtasks in data modeling. For 
example, a typical sub task in data modeling is achieving 
second normal form. Each subtask will have multiple 
production rules, depending on the expertise of the 
individual (Anderson, 1993). One possible production 
rule for this subtask would be: 

If an entity that is in first normal form possesses 
a compound primary key and if every nonkey 
attribute is fully functionally dependent on the 
primary key, then note that that entity is in 
second normal form. 

This productio n rule repre sents a sub goal for the overall 
task of achieving second normal form, namely the 
subgoal of identifying whether an entity is in second 
normal form. An individual would need to possess 
additional production rules for modifying an entity that 
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is not in second normal form so that it achieves second 
normal form. 

The following is an example of a question that could be 
asked to test whether an individual is able to identify 
entities in second normal form. 

Which of the following entities is in second normal 
form? 

a. An Employee entity that tracks the phone numbers 
that belong to each employee. There may be 
anywhere between 1 and 4 phone numbers. The 
attributes include Employee ID, Last Name, First 
Name, Department, and Phone Number. 

b. An Employee Skill entity that tracks the skills 
possessed by each employee. The attributes include 
Employee ID and Skill ID. 

c. An Employee Skill entity that tracks the skills 
possessedby each employee. The attributes include 
Employee ID, Skill ID and Skill Description. 

d. Both a and b. 

A total of 105 questions were written for the instrum ent. 
Multiple questions were written for each of the items 
listedin Appendix A except for the items concerning the 
use of a CASE tool. A single question was written for 
those items to limit the redundancy of the instrument. 
Five of the original questions were dropped because they 
appeared ambiguous or confusing in relation to other 
questions on the exam. An additional question was 
dropped because the answer was obvious, leaving a total 
of 99 questions. Forty-five of those questions tested 
declarative knowledge, and 54 questions tested 
procedural knowledge. The questions were rearranged 
so they appeared in random order. 

Subjects consisted of 60 undergraduate IS students at a 
large southeastern university in North America. The 
students were taken from two differentclasses: Systems 
Analysis and Design (23 students) and Database Design 
and Administration (37 students). Systems Analysis and 
Design is one of two initial courses in the IS major at 



this university and is a prerequisite to the Database 
Design and Administration course. At the time the 
instrument was administered, students in the Systems 
Analysis and Design course had just completed the 
classesthat introduced them to data modeling. Students 
in the Database Design and Administration course had 
completed the data modeling portion of the course when 
the instrument was administered. Therefore, students in 
both courses had received instruction on data modeling, 
but the students in the Database Design course had more 
experience in actually constructing data models. The 
students were given extra credit for participating in the 
study and they received individual reports after 
completing the examination that listed areas of data 
modeling where they had given incorrect answers. 

Students were not given advance notice about 
completing the instrument. All of the Systems Analysis 
and Design students completedthe instrument during a 
single class period. Most of the Database Design and 
Administration students (27) completed the instrument 
also during a single class period. A second section of the 
Database Design class was given the opportunity to 
complete the instrument outside of class. Ten students 
from the second section of the class completed the 
instrument under controlled conditions. No significant 
differences were found between the performance of 
those ten students and the rest of the Database Design 
students. Because the Systems Analysis and Database 
Design classes are taken at different times during the 
major, students in the two classes generally do not know 
each other. 

RESULTS 

Descriptive statistics for each class are presented in 
Table 2. It was unexpected that both classes would 
achieve similar averages; although the range for the 
Database Design class was slightly wider, with students 
achieving a higher maximum score. Table 3 presents 
descriptive statistics for the instrument broken down by 
the questions testing declarative and procedural 
knowledge. As hypothesized, the mean scores for the 
declarative questions are higher than the corresponding 
scores on the procedural questions. 
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TABLE 2 

DESCRIPTIVE STATISTICS FOR THE PERCENTAGE CORRECT BROKEN DOWN BY CLASS 





Average 


Standard 

Deviation 


Median 


Maximum 


Minimum 


SAAD 


• 57.00 


8.44 


58.00 


70.00 


42.00 


DDAA 


59.65 


9.29 


59.60 


80.81 


43.43 


Overall 


58.63 


8.99 


59.30 


80.81 


42.00 



TABLE 3 

DESCRIPTIVE STATISTICS FOR THE PERCENTAGECORRECT 
BROKEN DOWN BY TYPE OF QUESTION 





Average 


Standard 

Deviation 


Median 


Maximum 


Minimum 


Declarative 


68.04 


11.63 


68.89 


88.89 


44.44 


Procedural 


50.50 


9.56 


50.00 


74.07 


33.33 


Overall 


58.63 


8.99 


57.78 


88.89 


33.33 



The firsthypothes is was tested using a paired hypothesis 
test. For each of the 60 students, the percent correct for 
the procedural questions was compared to the percent 
correct for the declarative questio ns. The only assump- 
tion for this type of test is that the sample size is 
sufficiently large (Sincich, 1991). The sample size is 
based on the number of students that completed the 
instrument (60 students), which is well above the 
accepted guideline of a sample size of 30 (Sincich, 
1991). 

The next two hypothesis tests were large sample, two 
population mean hypothesis tests. For the second hypo- 
thesis test, the mean percentage correct on the 
declarative questions within one class was compared to 
the mean percentage correcton the declarative questions 
within the other class. The third hypothesis test was 
similar except that the mean percentage correct on the 
procedural questions was compared between the two 
classes. 

There are two assumptions associated with two 
population mean hypothesis tests (Mendenhall and 
Sincich, 1993): that both samples are randomly and 
independently selected, and that the two sample sizes are 
sufficiently large. The two classes were randomly 
selectedout of severalpossible sections offered foreach 
class during a semester. Because one class (Systems 
Analysis and Design) is a prerequisite of the other class, 
the population of students talcing one of the classes 



during a given semester should be a different set of 
students than the population ofstudents taking the other 
class during that semester. Thus, the selection ofstudents 
in each of the two classes was independent of the other. 
The sample size for the declarative questions is 45 and 
the sample size for the procedural questions is 54, 
indicating that the sample sizes are large enough 
(Sincich, 1991). Therefore, both of the assumptions 
appear to be satisfied. 

The results of the three hypothesis tests are summarized 
in Table 4. Because thisis an initial study, eachtest was 
performed at a 90% significance level. Based on the 
results of the first hypothesis test, it appears that there is 
a significant difference in the performance of subjects 
between the declarative and procedural questions. The 
second test failed to indicate a significant difference in 
the performance between the classes on declarative 
questions and the third test indicated that those subjects 
with more experience in data modeling performed better 
at the procedural questions. 

DISCUSSION AND LIMITATIONS 

Based on the results of the hypothesis tests presented in 
Table 4, it appears that there is a significant difference in 
the performance of subjects between the declarative and 
procedural questions and that those subjects with more 
experience in data modeling performed better at the 
procedural questions. These results provide initial 
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TABLE 4 

SUMMARY OF THE RESULTS OF THE HYPOTHESIS TESTS 



Hypothesis Test 


Test Statistic 


Significance 


Conclusion at a=0.10 


^la’ I^Dcc ^ I^Proc 


z= 12.51 


p = 0.00 


Reject H^ 


^2a- 1^0604212 ^ 1^0003113 


z = -0.46 


p = 0.68 


Fail to reject H^ 


^3a- M'Proc4212 ^ M'Proc3113 


z= 1.38 


p ^ 0.08 


Reject Hq 



evidence that the instrument can discriminate between 
the declarative and procedural types of knowledge. 

While it has been demonstrated that there is a significant 
difference in the performance of students on the two 
types of questions, it is still unclear whether this is 
because procedural knowledge is being evaluated. Other 
competing explanations for the difference in per- 
formance could include students’ un familiarity with this 
form of testing, or that the questions are testing two 
different constructs, not necessarily declarative and 
procedural knowledge. Future research could address 
this limitation by correlating student performance on this 
type of examination with their performance on actual 
data modeling tasks, or by comparing performance on 
this type of examination with standard forms of mental 
model measurement (Kraiger, Salas and Cannon- 
Bowers, 1995). 

Also, production rules were chosen randomly as the 
examination was developed. Future work could establish 
a more complete library of production rules within a 
cognitive skill domain. Instructors could then develop 
examinations using these libraries, ensuring that the 
complete skill domain is represented by questions on an 
examination. 

Evaluating procedural knowledge on an objective basis 
can potentially provide students and educators with a 
betterunderstanding of students’ acquisition ofIS skills. 
The results from this study are encouraging, but more 
work is clearly still needed. The limitations described 
above need to be addressed, and a better understanding 
ofevaluatingprocedural knowledge in IS students needs 
to be developed. 
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APPENDIX A 

DATA MODELING CONSTRUCTS AND PROCESSES 



1. Data model definition 

2. Need for data models 

3. Importance of data modeling within the systems 
development lifecycle 

4. Logical and physical data model definition 

5. Need for logical/physical data models 

6. Enterprise data model definition 

7. Need for enterprise data models 

8. Definitions of database models (e.g., hierarchical, 
network, relational) 

9. Historical use of database models 

10. Definition of entity relationship diagrams 

11. Definition of entities 

12. Definition of relationships 

13. Determining entities 

14. Determining which entities are related to each other 

15. Definition of cardinality 

16. Determining the cardinality of a relationship 

17. Definition of the degree of a relationship 

18. Determining the degree of a relationship 

19. Definition of an associative entity/gerund 

20. Determining entities that are gemnds 

21. Definition of categorization/generalization 

22. Definition of supertypes and subtypes 

23. Determining entities that are supertypes and 
subtypes 

24. Definition of a repository 

25. Need for a repository 

26. Definition of attributes 

27. Determining the attributes for an entity 

28. Definition of fields and records 

29. Definition of functional dependencies and 
determinants 

30. Identifying functional dependencies between 
attributes 

3 1 . Definition of primary keys 

32. Determining the primary keys for entities 

33. Definition of compound keys 

34. Identifying compound keys 

35. Definition of foreign keys 

36. Reasons for foreign keys 

37. Identifying foreign keys 

38. Definition of multivalued attributes 

39. Identifying multivalued attributes 

40. Resolving multivalued attributes 

41. Definition of aliases for attributes 

42. Need for aliases of attributes 

43. Definition of data type (in relation to attributes) 

* Items added by subject matter experts. 



44. Determining the data type of attributes 

45. Definition of domains (limits and range checks for 
attributes) 

46. Determining the domain of an attribute 

47. Definition of normalization 

48. Reasons for normalization 

49. Definition of first normal form 

50. Achieving first normal form 

51. Definition of second normal form 

52. Achieving second normal form 

53. Definition of third normal form 

54. Achieving third normal form 

55. Definition of fourth normal form 

56. Achieving fourth normal form 

57. Definition of fifth normal form 

58. Achieving fifth normal form 

59. Definition of Boyce-Codd normal form 

60. Achieving Boyce-Codd normal form 

61. Definition of domain-key normal form 

62. Achieving domain-key normal form 

63. Definition of CASE tools 

64. Uses for CASE tools 

65. Starting a new project in a CASE tool 

66. Starting a new ERD in a CASE tool 

67. Placing entities on a diagram in a CASE tool 

68. Describing relationships between entities on a 
diagram in a CASE tool 

69. Saving an ERD in a CASE tool 

70. Printing an ERD in a CASE tool 

71. Analyzing an ERD for syntactical errors using a 
CASE tool 

72. Finding the repository in a CASE tool 

73. Navigating through the repository in a CASE tool 

74. Describing the data types of attributes in the 
repository of a CASE tool 

75. Describing the domains of attributes in the 
repository of a CASE tool 

76. Definition of business rules 

77. Reasons for documenting business rules 

78. Documenting business rules in a CASE tool 

79. Checking that all key information is correctly 
identified for a data model 

80. Generating SQL code from an ERD and repository 
in a CASE tool 

81. Printing reports on the repository in a CASE tool 

82. Need for domains* 

83. Adding attributes in a CASE tool repository* 
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APPENDIX B 

RESULTS OF DATA MODELING SURVEY, RANKED BY MEDIAN VALUE 



Importance 


Item 


Mean 


Median 


10 


4.67 


5 


II 


4.50 


5 


12 


4.67 


5 


13 


4.50 


5 


14 


4.67 


5 


17 


4.50 


5 


18 


4.67 


5 


19 


5.00 


5 


20 


4.83 


5 


24 


4.33 


5 


26 


4.67 


5 


27 


4.67 


5 


31 


4.67 


5 


32 


4.83 


5 


36 


4.67 


5 


37 


4.67 


5 


52 


4.17 


5 


54 


4.17 


5 


63 


4.33 


5 


64 


4.83 


5 


65 


4.67 


5 


66 


4.67 


5 


67 


4.67 


5 


68 


4.67 


5 


15 


4.33 


4.5 


16 


4.50 


4.5 


25 


4.33 


4.5 


28 


4.00 


4.5 


33 


4.50 


4.5 


34 


4.50 


4.5 


35 


4.50 


4.5 


39 


4.50 


4.5 


43 


4.33 


4.5 


44 


4.33 


4.5 


47 


3.83 


4.5 


48 


3.83 


4.5 


49 


3.83 


4.5 


50 


4.00 


4.5 


51 


3.83 


4.5 


53 


3.83 


4.5 


69 


4.33 


4.5 



Importance 


Item 


Mean 


Median 


70 


4.17 


4.5 


71 


4.33 


4.5 


72 


4.33 


4.5 


74 


4.33 


4.5 


75 


4.33 


4.5 


81 


4.00 


4.5 


4 


4.00 


4 


21 


3.67 


4 


22 


3.83 


4 


23 


3.83 


4 


38 


4.33 


4 


40 


3.83 


4 


45 


4.00 


4 


46 


4.00 


4 


73 


4.00 


4 


77 


3.67 


4 


79 


3.40 


4 


I 


3.50 


3.5 


5 


3.83 


3.5 


76 


3.50 


3.5 


78 


3.33 


3.5 


2 


3.67 


3 


3 


3.67 


3 


6 


2.67 


3 


7 


2.67 


3 


29 


2.67 


3 


30 


2.83 


3 


41 


3.33 


3 


42 


3.17 


3 


8 


2.33 


2.5 


80 


2.67 


2.5 


9 


2.33 


2 


55 


1.83 


2 


56 


2.00 


2 


• 59 


2.00 


2 


60 


2.17 


2 


57 


1.67 


1.5 


58 


1.83 


1.5 


61 


1.83 


1.5 


62 


2.00 


1.5 
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ABSTRACT 

Technology and the associated scientific theory do not contain within themselves a theoretical base to guide if why 
and how they might be wisely used (other than, of course, technical instructions). Ethics units in the IS curriculum 
are required to help prom ote and develop appropriate human wisdom, particularly in a technological culture where 
mechanistic conceptions may trivialise the dignity of the human person. Wisdom in the Platonic sense, is knowledge 
of what is, of what is real, of what actually is the case. It is knowledge of the real world as distinctfrom the world of 
the legal, of conventions, or of appearances, the world which appears to be the case. In its practical aspects (ethics) 
it is a knowledge of the purpose of human life, human happiness and well being, and of the means to attain that end. 
In its speculative aspects, it views all that exists as a rationally ordered and harmonious whole, Socrates is wise in 
that he is committed to the world which is the case. In this he is involved in the pursuit of wisdom and may be truly 
called aphilosopher. We therefore argue thatwithoutan ethicalperspective, various humanly devaluing philosophies 
may be accepted by default. Detectable, for example, in technological thinking is the materialist philosophy of 
mechanism. In fact, by default, science often assumes a mechanistic philosophy leading to a materialism devoid of 
ethics, which grew out of the origins of modern science. The discourse of artificial intelligence studied in IS units is 
used to illustrate these issues. 



INTRODUCTION 

Though not always clearly defined, theconcepts of data, 
information, and knowledge are central to the design of 
Information Systems (IS). They represent an ascending 
order of complexity where data consisting of raw facts 
become meaningful as patterns in blocks of data and so 
information expressed as propositions. When proposi- 
tions are meaningfully joined, as in cause-and-efifect 
relations, knowledge is produced. When present in an 
information system, data and, information exist without 
meaning; meaning becomes present only to the rational 
observer. Beyond these there is the further concept of 
wisdom which exists both as a criteria for the use of IS 
in its various forms, and as an essential quality of an 



educated person. Most generally, wisdom refers to the 
capacity of decision makers to recognise and apply 
values in such a humane way that human beings and 
their environment to flourish. Thus, wisdom also 
includes an awareness of the inestimable dignity ofeach 
human person beyond reason or merit, possessing 
inalienable rights, being ends in themselves, and 
experiencing inner lives rich in feelings, memories, 
thoughts and emotions. 

The paper argues for the place of ethics in the IS 
curriculum to detect humanly devaluing philosophies 
underlying the discourse and claims of technology. The 
project of artificial intelligence (AI) is used as a case 
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study as the technology makes many assumptions about 
its ultimate goals and the nature of the human person. 

WISDOM 

Wisdom is a concept originating in the ancient Near 
East. Egyptian examples of w isdom literature date back 
to 3"^* millennium BC, whilst the Hebrew wisdom 
movement flourished under King Solomon (961-922 
BC). Wisdom was thought to be an acquired quality or 
special kind of knowing about life’s meaning applicable 
to the successful living of daily life and attainable by 
those who seek it by patient reflection on experience. In 
another mystical sense it is an intuitive form of 
knowledge beyond the effort of thought, attainable as a 
gift by those who desire it. Wisdom assumes importance 
in the Western tradition of philosophy in the life of 
Socrates (470-399 BC). 

The wisdom, which the philosopher seeks, relates 
especially to the Socratic doctrine that virtue is 
knowledge. This means that none could realise the best 
in themselves and thereby become happy unless they 
knew clearly, as is appropriate for a rational being, what 
life is worth living for (Cornford, PI 75), the true 
meaning of life. This is provided by the highest object of 
knowledge, the essential nature or form of the “Good,” 
from which everything that is good and right derives its 
value for us (Republic, vi 504), and which is the final 
cause of all that is good in the universe, even of its very 
existence. From the vantage-pointof a knowledge of the 
Good, the whole of reality, including the moral and 
physical order is seen as an objective rational system 
(Cornford, P212). As the object of a purpose attributed 
to a divine Reason operating in the world, this supreme 
Good makes the world intelligible, just as a work of 
human craftsmanship becomes intelligible when we see 
the purpose it is designed to serve. Such knowledge, 
once attained, cannot fail to determine will and action 
since all desire what is good. 

The essence of the doctrine of forms (whatever of 
Plato’s two world’s mode of expressing it) is the 
conviction (showing remarkable similarity to the Biblical 
account) th at: 

• the world is good as its very existence as all that 
exists partakes of the form of the Good; 

• the world and all thatexists is ordered and is thereby 
intelligible; 

• life has a purpose which is knowledge or wisdom of 
. the Good, and living accordingly; 



• there is a final justice since “nothing can harm a 
good man, either in life or in death; nor are his 
fortunes neglected by the gods” (Aplogy, 41d); 

• there is a unique dignity for the human person 
among the things which exist given that “the soul is 
immortal and never perishes” (Republic, x, 608d); 

• the differences between good and evil, right and 
wrong, true and false, beautiful and ugly, are 
absolute, not relative to the customs or tastes or 
desires of individuals or social groups (Cornford, 
pl80). 

• there are objective and necessary truths. Good and 
evil, and so ethics, are discovered by rational 
reflection by the intellect rather than invented; they 
are not of our creation as are the legal or 
conventional and the illegal; 

Wisdom is here understood, therefore, to be a view of 
the world that assumes that there is an overall intelligible 
order in the universe which hasmeaning for human life, 
and has some form of goodness as its basis. Thus, there 
is to be found a harmony between all aspects of human 
existence including thought and behaviour, and the 
underlying first principles of the cosmos. There is, 
further, a fundamental dignity attributed to the human 
person that exists beyond all reason and merit. 

While ethical discourse tends to focus on the distinction 
between ethical andunethical behaviour, with a view to 
avoiding the latter, wisdom is further understood to 
imply an ability and desire to choose, from among 
ethical behaviours. That is, a mode of behaviour which 
might best respect the true meaning of human existence 
and so promote flourishing of the dignity of all human 
persons, 

APPLICATION OF WISDOM TO IS 

Philosophers, who commit themselves with passion to 
seek the Good and who are necessarily few in number, 
are those most appropriate to rule in the ideal state 
(P231), In the philosopher king of Plato’s Republic, 
there would be union of political power and love of 
wisdom. In such a person there will be the supremacy of 
reason, the divine element in man, over the rest of our 
nature, (P176) Rather than being free to do whatever he 
liked, the philosopher king would be like an artist 
working with constantreference to an unchanging model 
(the forms such as justice, goodness, and temperance) 
which irrevocably determines the outline and basic 
principles of his work (P 205), In contemplating a w orld 
of unchanging and harmonious order, where reason 
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governs and nothing can do orsuffer wrong; he will not 
fail to fashion himself in its likeness. 

For our own times, Plato’s mlers might well include 
those who provide the world with computer technology 
and other forms of mass communication which have so 
transformed modern living beyond the powers of 
national governing bodies. The need for philosopher- 
kings might now equate with the need for a guiding 
philosophy for those constructing, using and discoursing 
on the artefacts of technology; they also need to be 
philosophers in'the Platonic sense ofbeing committed to 
the pursuit of wisdom, 

ETHIC UNITS IN THE IS CURRICULUM 

In its practical aspects, wisdom, assuming an objective 
moral order, gives direction to human practical and 
ethical activities, provides an ability to order one’s life 
well, and leads to a good and happy life. In its 
speculative aspects wisdom strives to grasp first 
principles, asking the metaphysical “why” questions. It 
judges and orders less universal disciplines. Science, for 
example, simply accepts rather than critiques its own 
first principles (assumptions about an orderly universe 
and about human knowing), Neitherdoes science advise 
on the proper or best use of the technology to which it 
gives rise. 

Technology and the associated scientific theory do not 
contain within themselves a theoretical base to guide if, 
why and how they might be wise ly'used (other than, of 
course, technical instructions). Ethics units in the IS 
curriculum are required to help promote and develop 
appropriate human wisdom, particularly in a 
technological culture where mechanistic conceptions 
(what appears to be the case) may trivialise the dignity 
of the human person. Without an ethical perspective, 
various humanly devaluing philosophies may be 
accepted by default. Detectable, for example, in 
technological thinking is the materialist philosophy of 
mechanism. In fact, by default, science often assumes a 
mechanistic philosophy leading to amaterialism devo id 
of ethics, which grew out of the origins of modern 
science, 

MECHANISM: 

WHAT SEEMS TO BE THE CASE 

Modern science finds its origins in the 16^*' and 17^’’ 
centuries. During this significant period, Bacon (1588- 
1679) proposes the scientific method, Galileo (1564- 



1632) explains natural phenomena (using efficient 
causes and matter in motion), Descartes (1594-1650) 
employs the method of mathematics (requiring clear and 
simple ideas as axioms), and Newton (1632-1704) 
provides a comprehensive system of mechanics based on 
mathematical laws governing the behaviour of 
conceptual models (particles with mass concentrated at 
a point), 

Newton discoveredthatthe movements of natural bodies 
approximated those predicted by these mathematical 
laws and mechanistic models, which were therefore 
intelligible, discoverable anduseful forprediction. In the 
modelling process complex natural bodies, are 
considered - in a process of reduction - from a single 
viewpoint: the quantitative ormathematical. The natural 
physical body (what is the case), which cannot be fully 
comprehended in its multidimensional richness, is 
reduced to the relatively familiar and comprehensible 
point-object particle in motion (what seems to be the 
case). The driving force of mechanism with its 
accompanying use of efficientcausality eventually ledto 
the spectacular success enjoyed by the Industrial 
Revolution, 

MECHANISM AS MATERIALISM 

Mechanism becomes a metaphysics when the 
assumption is made that all phenomena, including 
natural bodies, are adequately explained by intrinsically 
immutable (i.e. subject to no intrinsic change) quantity 
and local motion, the basic characteristics of machines. 
This thinking finds its origins in the 'one and the many* 
debate between Parmenides and Heraclitus* in ancient 
Greece (Cavendish, 1 985), seemingly resolved by 
Democritus who posited the existence of tiny 
indestructible material units (the one or being) called 
atoms and a void (non-being) allowing forplurality and 
motion (Torretti, 1999, pl5). The atoms are therefore 
able to combine in various ways to form the changing 
macroscopic bodies of sense observation (the many). All 
phenomena, including natural bodies were then to be 
explained in terms of quantity and local motion. Whilst 
being derived from, although notfully defining, physical 
objects these models can be considered separately from 
the objects, although they can not require a separate 
existence simply by virtue ofbeing intelligible, 

Hobbes (1588-1679), whose long life spanned the 
modern origins of science, is considered the founder of 
modern metaphysical materialism (Flew, 1985), He 
equated the quantitative models with the whole of 
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reality, ignoring the reductive process involved in their 
origin, and so created a metaphysics of mechanism. To 
identify models (eg. Newtonian particles) with natural 
bodies, or to regard them as equivalent, is to commit a 
category mistake of a type identified by Wittgenstein. 
Although they can appear in sentences of the same 
logicalform, naturalbodiesandmathematical models do 
not enjoy the same form of existence. 

Upon such a metaphysics, Hobbes developed an 
epistemology as well as natural, moral and civil 
philosophies which have continued to influence thinking 
to this day. Mechanism has become the common-sense 
method of understanding all physical phenomena. 
Furthermore, human systems such as politics and 
economics may also be thought to operate as machines 
with individuals mere cogs unable to determine their 
future. 

Descartes(l 596-1650), inhis foundational search forthe 
same absolute clarity and definiteness from the physical 
world as is found in mathematics, gained an apparently 
certain basis for knowledge from the existence of his 
mind which thinks (Watling, 1985). Distinguishing his 
mind or thinking self from his body, he placed the 
essence of bodies in their extension (composed of 
integral parts) with local motion as the only motion 
considered. This mind-body problem became a mind- 
matter problem when, as explained above, the abstract 
mode Is were assumed to be existing physical entities, a 
process suited to the rationalist thinking ofDescartes as 
it provided the desirable clear and simple ideas upon 
which a system of knowledge may be built. It was then 
an easy step for Hobbes to deny mind as a separate 
substance, and to make mind and matter equivalent. 
Mind then is considered to be fully explicable in terms 
of the mathematical models become physical particles, 
and motion. 

MECHANISM AND AI 

Such a conception of the human mind finds sympathy in 
contempo rary attempts to construct artificial intelligence 
(AI) where some researchers “maintain that suitably 
programmed compu ters can literally be said to engage in 
processes of thought and reasoning” (Lowe, 2000, p 1 93) 
thus emulating high-level functions of the rational 
human person. In AI, rather than the mathematical 
models of mechanics, the primary data are formal 
symbols embodied in an electronic memory device. 
Physical laws are replaced by the syntax of coded logical 
rules which manipulate the formal symbols under the 



power of a processing unit, according to the procedures 
built into these mles in order to simulate or model the 
computational processes of the human mind. In this 
understan ding of AI (which is termed weak AI by Searle, 

1 990) the objectives include both development of more 
powerful mind-simulation programs and an improved 
understandin g of the worki ngs of th e hum an mi nd. 

The reductionist process of mechanism, again making a 
category mistake and identifying the model (here 
symbols and the processing unit) with the natural body 
(mind of the human person), leadsto the claim (which is 
termed strong AI by Se arle, 1 990) th at it will eventu ally 
be possible to create a mind, equivalent to a human 
mind, simply by designing a sufficiently complex 
computer program with the right inputs, logical 
procedures, and outputs. It is claimed that although “the 
human mind is complex, yet in principle it is equivalent 
to a finite automaton, which may be constructed from 
logical nan's (or nor's) and erasable bit memories” 
(Burks, 1990, p469). Popularpublicationscontain claims 
such as “researchers from a variety of fields are.... 
predicting the creation of truly intelligent machines 
sometime this century” (Gallagher, 2000). It is not, then, 
a great leap from developing technology that imitates 
certain aspects ofthe functioning of the human brain, to 
concluding that the machine can think by virtue of 
implementing a computer program, and conversely that 
the human person is nothingbut a machine, albeit a very 
complex one. 

The Turing Test is designed to provide a scientific 
measure of the success of AI (producing conscious 
intelligence), by making a furtherreduction in assuming 
that if something behaves as if it had certain mental 
processes, then it must actually have those processes. 
However, the Turing test measures only the observable. 
Consciousness and knowing are not directly observable 
by others but remain a part of my experience. The very 
language used in the theory of AI and in this test 
suggests a certain equality of status for a computer 
system exhibiting certain types of behaviour (what 
appears to be the case) and the nature of the human 
person (what is the case). Further, the philosophy of 
logical mechanism holds, as a central thesis, that a finite 
deterministic automaton can perform all human 
functions (Burks, 1990, p409). 

ETHICAL IMPLICATIONS OF MECHANISM 

If the human person is fully explicable in terms of 
behaviour there is, according to behaviourist theory (eg.. 
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Skinner, 1973, chp 6), only caused behaviour which has 
remote causes in evolutionary history resulting in genetic 
endow ment, and prox imal cau ses in the fee dback effect, 
known as operant conditioning, of the consequences of 
behaviour. These latter are the reinforcers or 
punishments, things which please or displease when 
experienced and which are thereby classified as good or 
bad as value judgements are made. Human life and 
behaviour are simply a continuation of the evolutionary 
process of successful adaptation to a changing 
environment. Positive reinforcers are such because they 
led to advantages in terms of species survival and are 
incorporated into species genetic make up. 

Thus, there can be no objective basis for a theory of 
ethics that discovers the good to be done based on 
human dignity and purpose. From a mechanistic 
viewpoint there is no objective human dignity, just the 
appearance of it where the causes of altmistic behaviour 
are inconspicuous. The human person does not have an 
objective nature upon which appropriate behaviour may 
be judged, only an evolutionary history determined as 
the species adapts to the environment. Further, there is 
no need for ethics; desirable (adaptive) behaviour will 
follow deterministically when the consequences of 
behaviour are allowed to be experienced. 

The only logical ethics to be applied is that of 
utilitarianism which makes minimal assumptions about 
the human person. This involves the maximisation of 
good (understood as pleasure) and the minimisation of 
harm (understood as pain) for the greatest number. 
Personal freedom can be exercised to the extent that no 
harm is done to anyone. 

It should further be noted that ideas also have social 
consequences. If human persons are fully explicable in 
mechanistic terms, then they mightjustifiably be treated 
as objects (machines) to be used, or as means to ends. 
Just as items of computer technology may be readily 
replaced, without ethical concern, in the financial 
interests of shareholders, so also may workers be 
disposed of for similar reasons. Even a cursory 
acquaintance with contemporary social problems (eg. job 
losses and rural decline brought about by bank branch 
closures resulting from network communications) 
reveals that such scenarios are no longer simply 
theoretical. 



CRITIQUE OF MECHANISM 

There would seem to be convincing evidence that there 
is more to the explanation of natural bodies than can be 
provided by the mathematical models and abstractions of 
Newtonian Physics. Features of modern physics make a 
philosophical position that equates a natural body with 
its mathem atical model less tenable. It can be argued that 
a position that equates mental processes such as 
reasoning with a computational model of reasoning is 
similarly lacking in justification . Computational models 
of mental processes (in terms of exactness of function 
and reflection) are no more real than the computational 
models of any other natural phenomena. 

20™ CENTURY PHYSICS 

The arbitrariness of the adoption of a mechanistic 
philosophy should be noted, since it has long been 
known that mechanistic theories, so successful in 
Newtonian physics are no longer compatible with many 
aspects of 20^*^ century physics where many common 
sense understandings are found to be inadequate. 
Ordinary understandings of space and time are 
undermined (Theory of Relativity), indestructible and 
unchanging particles behave as waves obeying laws of 
chance in certain experiments (quantum physics), and 
the simple laws of Newtonian Physics often fail to 
explain the often capricious and haphazard behaviour of 
some natural systems (chaos theory). 

HUMAN COGNITION 

The central thesis of this paper is that there is an 
essential non-material aspect to human rationality. 
Natural bodies (things) are assumed to exist. The human 
person may find intelligible the essential nature of the 
thing by a process of abstracting (non-exhaustively, of 
course) from appearances (cf. Kretzmann & Stump, 
1993, pl42). The essential nature contains those features 
involved in a definition (eventually in terms of self- 
evident truths rather than involving an infinite regress) 
which cannot be changed without giving rise to a 
different kind of thing. The result of this abstraction is 
the concept or idea, which is separate from, but finds 
expression in words or speech. The possession of ideas 
is proposed as a pre-requisite for intelligence, and is 
what is meant by “knowing”. Reasoning is then the 
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process of drawing conclusions from propositions whose 
meaning is understood, because the idea corresponding 
to each word is possessed, In this process, a sequence of 
propositions are related according to the laws of logic, 
by means of cause and effect, or means-to-ends 
relationships, 

COMPUTER COGNITION 

That possessing ideas or knowing, the essential feature 
of human cognition and reasoning is lackingin computer 
cognition is well demonstrated by the Chinese room 
argument (Searle, 1990). This distinguishes mental 
contents biologically produced by the brain (semantics 
for Searle, or knowing ideas leading to meaning in the 
previous discussion) and the formal symbol 
manipulation done by a computer (syntax for Searle, or 
reasoning without knowing). 

This argument considers a person with no understanding 
of the Chinese language alone in a room, possessing only 
a manual for relating o ne Chinese sym bol to another (cf, 
a computer pro gram), and receiving questions written in 
Chinese (input data) from outside the room. Using the 
manual, answers are composed by matching new 
symbols with symbols in the question, and then handed 
back to those outside the room (output), such that when 
the Turing test is applied, the answers provided are 
indistinguishable from those of a native Chinese speaker. 

Yet, like the computer, the person in the room merely 
manipulates symbols to which no meaning (idea or 
knowing) is attached, before or after the exercise. 
Further, this person knows that she is merely 
manipulatingsymbols, without any understanding ofthe 
symbol sets. The symbol manipulation occurs without 
reference to any meanings, does not give access to the 
meaning of the symbols, and so does not involve 
cognition. Similarly, computer programs have syntax 
only, but no semantics; the mere manipulation of 
symbols is not equivalent to human reasoning since ideas 
and therefore knowing are absent. 

If our interpreter were later to gain a mastery of the 
Chinese language and then re-visit the room, her 
experience would be entirely different, akin to switching 
on a light bulb in an unknown darkened room. She 
would now know the meanings of questions, and know 
thatshe now knows. Ratherthan the static answers ofthe 
manual, the answers to a given question might now vary 
over time as the interpreter's knowledge and experience 
grow. 



HUMAN CONSCIOUSNESS AS 
SUBJECTIVE EXPERIENCE 

The elusive concept of human consciousness has been 
described as the subjective character of experience 
(Nagel, 1974). For any organism, this is what it is for the 
organism to be that organism, the mental state or inner 
life of the organism . In the position taken in the paper, 
human consciousness would include the sum total of all 
knowing in terms of ideas possessed (however 
inadequate and awaiting further development) 
particularly in the context of the experience ofknowing 
that one knows leading to awareness. One is also 
conscious (has knowledge) of otheraspects ofthe inner 
life such as feelings and emotions. 

To develop any model of consciousness (to elucidate 
understanding or build an objective theory), there must 
be a reduction from multiple viewpoints to a single 
viewpoint; some aspects must be left aside. However, 
there is only a single point of view in consciousness, the 
subjective character of the experience (what it is like for 
a bat to be, rather than to behave as, a bat, to use Nagel's 
analogy), and so any sort of reduction is not possible. 
The phenomenological features of experience or 
consciousness cannot be excluded in a reduction (as is 
usually the case in deriving models) because that is all 
there is to experience. An analysis in ternis of functional 
and intentional states is not possible, because robots 
have these but since they do not know as we have 
definedknowing they do not experience states or events. 

Ultimately, it is not possible to produce a physical theory 
of the mind (to be distinguished from physical processes 
inthebrain). “If the facts ofexperience, facts about what 
it is like for the experiencing organism are accessible 
from only one point of view, then it is a mystery how the 
true character of experience could be revealed in the 
physical operationof the organism" (N agel, 1974, p385). 
It follows the n that it could not be possible, by electronic 
devices such as a robot, to produce or even emulate 
intelligent human consciousness, and so produce the 
fully functioning human person. That which cannot be 
modelled, cannot be symbolically represented in a 
computer program. 

HUMAN AND COMPUTATIONAL REASONING 

Electronic computers are ordered towards ends orgoals, 
as these are set by a code writer listing instmctions. AI 
technologies are essentially based on computers 
operating according to software instructions and are fully 
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explicable as goal-directed, adaptive feedback systems 
which have: 

• inputs (eg. present scenario for chess software, 
characteristic data for artificial neural nets (ANN), 
uncertainty estimates for fuzzy expert systems(ES)), 

• deterministic processes (eg. rules of the game for 
chess, mathematical model for ANN’s, inference 
rules for fuzzy logic) which lead to, 

• an output to be compared with a predetermined goal 
(best move in chess gam e, output data to be matched 
in ANN’s, conclusion best in accord with data for 
fuzzy ES) to give an error, and finally, 

• Negative feedback which operates in an iterative 
step-by-step procedure to reduce the error by 
altering aspects of the process (point of search for 
chess, adjusting parameters in a "learning" process 
for ANN’s) or aspects of the input data (fuzzy ES). 

By definition, AI processes, as with other computer 
processes, do not step outside the well-defined step-by- 
step procedures of the programmer’s code to think for 
themselves by understanding previously mentioned 
cause and effect or means-to-ends relationships. 

Computer programs manipulate symbols (and the 
quantities they represent) according to procedures 
designedby the programmer who knows meaningand so 
possessesintelligence. The program m ay be perceived to 
behave as though it were intelligent, but only by virtue 
ofthe programmer who knows, and who must, therefore, 
as is widely acknowledged, accept responsibility for the 
performance of a particular device. The set ofprocedures 
embodied in executing instruction code may be 
perceived to involve reasoning, but again only by virtue 
of the reasoning powers of the programmer who knows 
the meaning of propositions, because she knows the 
meanings ofthe related concepts, and is able to constmct 
the algorithm. This does not, of course, preclude the 
possibility of algorithms which might search for 
solutions under heu ristic constraints, or discover proofs 
for theorems. Furtherpossibilities such as program self- 
modification resulting from “learning” (eg. from an error 
calculation), or even program or computer reproduction 
may, again, only be present by virtue ofthe code. 

ANN’s imitate something of what is known of the 
cellular structure and physiology of the human brain. 
Stimulus (input) - response (desired outcome) training 



builds appropriate numerical values for weights and 
biases on mathematical mode Is of neurons using training 
data. Purely mechanical explanations describe the 
predictive (“intelligent” or “learned”) behaviour of the 
resulting mathematical expression. Rational activity of 
the human person cannot, therefore, be equated with the 
training environment of ANN’s, or the programmed 
activity of Expert Systems. 

Robots respond to stimuli from sensors according to a 
pre-arranged plan determined by the programmer and 
incorporated into a program code. The received sensor 
information is limited to material stimulation from the 
physical environment. Thus, such holistic and 
unquantifiable qualities as truth, beauty, harmony, 
proportion and brilliance which profoundly move the 
knowing human person when experienced in a work of 
great art, for example, will not be accessible to a robot. 

Computers lack knowledge of relations, particularly 
those of cause-and-effect, as distinct from production 
rules, however complex, which are included in the 
coding for an Expert System. These latter are a function 
of the knowledge of relations possessed by the expert, 
acquired by the knowledge engineer, and coded by the 
system builder. A robotprogrammed or trained to return 
a certain tool will not look around for a suitable 
replacement when the tool is missing (unless, of course, 
specifically programmed to do so on a set of 
probabilistic usefulness functions). Even then the 
returnedreplacement will followthe programmer’s code, 
rather than be the result ofthe robot’s understanding of 
the function of the tool. 

Computers lack human progress in the sense that they 
lack the property of self-awareness and reflection 
characteristic of human intelligence. This is to be seen as 
distinct from the ability to record (and so be trained by) 
and respond to past patterns of user behaviour. The 
computer is unable to reflect on its own performan ce, to 
know that it knows as does a human person. The 
computer, as already noted, can never step outside the 
code, reflect on the code, and contribute its own 
observations. 

Computers may be programmed to make sounds 
imitative of a language in response to electronic stimuli, 
but lack speech since they lack concepts or ideas that 
find expression in words. Conversing can only mean 
making sounds without awareness of meaning. Thus, 
there is nothing possessed by the computer outside the 
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program code (such as concepts or ideas), that 
corresponds to the sounds imitating human speech. 

From intelligence in human persons there necessarily 
follows freedom of choice and the possibility ofethical 
life, but neither of these is found in computers. All 
choices are determined by the driving code, even if this 
code includes the generation of data from a probability 
distribution as in Monte-Carlo simulation. In simulation, 
a chance element is introduced into individual choices, 
but the over-all pattern of the probability distribution 
provides a deterministic outcome. 

The absence of the possibility of their own ethical life 
means that electronic machines, or in fact any artefacts 
oftechnology, do notcontain within themselves answers 
to questions about their possible ethical use, that is their 
use in such a way as to enhance the dignity ofall human 
beings they are designed to serve. Instruction manuals 
explain how to operate computer system s, but answers to 
questions such as how, when, why and for what purpose 
various systems should be used may well be/ethical 
questions with answers derivable from a values 
framework. 

HOLISTIC CONCEPTION 
OF THE HUMAN PERSON 

We propose an holistic and so non-reductionist (claiming 
to be what is the case) description of the human person 
as a basis for human ethics, from which flows essential 
human dignity as a basis for an objective human ethics. 
This description includes such exclusively human 
features we perceive in ourselves, all arguably arising 
from the uniquely human possibility of knowing, and 
knowing that we know, leading to consciousness, as the 
following: 

• Rationality and desire for understanding and 
meaning, despite the acknowledged limitations 
imposed by unconscious psychological drives (cf. 
the work of Freud, Jung); 

• Command oflanguage enabling communication and 
exchange of ideas (thoughts), feelings, confidences, 
joys and happiness; 

• Reflectiveconsciousness or self-awareness; not only 
knowing, but knowing that we know; 

• Subjectiv ity - a subject (thinker) as distinct from an 
object (which is thought about) or a thing; 
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• Incommunicable individuality; 

• Creativity as found in the poet, writer, artist, often 
seemingly driven to create by a power greater than 
themselves; 

• Possessing a centre of freedom whereby a person 
determines to choose (ratherthan is determined by) 
one motive among many, and so a particular course 
of action. Correlative to freedom is awareness of 
responsibility, the need to account for one’s actions; 

• Awareness of good to be done and evil to be avoided 
(cf the work of Kant drawing attention to the moral 
law within); 

• Non-rational experiences such as emotions, desires, 
feelings, intuitions, dreams and images, 
premonitions, experience of synchronicity or 
meaningful coincidences which seem to be due to 
more than chance, including chance escapes from 
disaster, a mystical sense of awe at the complexity 
and beauty of all that is observed in the natural 
world, a sense of the sacred even in the ordinary, a 
sense of a spiritual character or soul; 

• Bodiliness as a centre for relationships, sensuality 
and sexuality, distinguishing male and female, 
possessing privacy and dignity; 

• Mortality as being subject to the certainty of death, 
but the subsequent anxiety being transcended by a 
desire for a higher life. 

These aspects of the human person which, like the 
consciousness described above, defy mechanical or 
mathematical descriptions have traditionally been 
embodied in the concept of the soul. 

ETHICAL IMPLICATIONS 

These characteristics of the hum an person are the source 
of a special dignity. Like consciousness, described 
above, they cannot be reduced to an objective nature, 
using mathematical or other models. Conversely they 
cannot be reproduced using electronics to activate 
models embodied in computer programs. All this is 
arguably trivialised by mechanistic conceptions of the 
human person. The ethical critique should, therefore, 
identify the limitations of mechanism as a materialist 
philosophy which, whatever its value, is adopted or 
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imposed by, but does not originate within computer 
science. 

However, perhaps the most important source of human 
dignity, moral worth and title to respect arises from the 
responses we make to other persons and the way we 
matter to one another expressed as “love in its many 
genuine forms, ...from sexual love to the impartial love 
of saints” without which we would not have “a sense of 
the sacredness of individuals oroftheir inalienable rights 
or dignity. ..a sense that human beings are precious 
beyond reason or merit” (Gaita, 1999, p 5,8). W e come to 
know, firstly and in a special way in those we love, an 
inestimable dignity beyond reason (when there may 
seem no source of dignity as in those with severe mental 
or other disabilities) ormerit (where our love ortrusthas 
been seriously betrayed) possessing inalienable rights, 
being ends in themselves, and experiencing internal lives 
as rich in feelings, memories, thoughts and emotions as 
we find in ourselves. Respecting such dignity forms the 
objective basis for ethical treatment of others, made 
evident by the actions of those who love them, forms an 
objective basis for ethics. 

CONCLUSION 

Without, of course, questioning the value of AI for 
human welfare and progress, the paper has drawn 
attention to aspects of the discourse used in AI and 
someof its more controversial claims (particularly those 
of strong AI) which require understanding oftheir origin 
and an ethical critique. A philosophy of mechanism 
which makes the category mistake of giving the 
mathematical models of scientific mechanism the same 
sort of existence as natural bodies has been discussed. 
All reality, including human minds, are assumed to be 
explicable by material point objects exerting efficient 
causality of push-pull by virtue of their characteristic 
motion. If this is the basis of human rationality, the 
definingcharacteristic of the human person, then there is 
only human behaviour fully explicable in observable 
causes. This leaves no basis for “oughtness” in human 
behaviour; there is only that which pleases and that 
which displeases. There is no objective basis for an ethic 
of human behaviour, only a constantly adapting and 
evolving organism. 

Ethics units in the IS curriculum are able to establish an 
objective basis for ethics a holistic understanding of the 
person replaces the reductionist models and the 
inestimable dignity of the human person is established 
and protected. The holistic basis, trivialised by 



mechanistic conceptions, arises from consciousness and 
this “like life itself and time and free-will, is one of the 
great mysteries before which we have to fall silent” 
(Townsend, 2000). 

ENDNOTE 

1 . Each of whom performed reductions from the world 
of permanence and change in natural bodies (Heraclitus 
finding only measurable change, and Parmenides only 
the permanence ofbeing) then equated all of reality with 
their partial viewpoints. 
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ABSTRACT 

This paper describes the problems facing Information Systems educators in keeping current and producing graduates 
with skills that are valuable to employers. These problems are often magnified by rapid changes in technology. This 
paper proposes solutions to these problems through the creation of partnerships and learning alliances between 
industry and education. Several types of partnerships are described and reinforced with examples. These include: 
advisory committees, student internships, industry projects for college seniors, faculty internships/consulting, 
faculty/industry exchange programs , faculty training and updating, hardware/software resource sharing, mentoring, 
research grants and contracts. 



INTRODUCTION 

The many advancements in computer technology in the 
last few years, both hardware and software, have brought 
new challenges and opportunities to industry and 
academia alike. As most computer and information 
technology professionals are aware, the useful life of 
computer hardware was measured in years during the 
1950s, '60s, and even into the '70s; but we now find new 
models of computers being introduced almost every 
month. Systems design methodologies and software 
development tools are also changing rapidly. "Because 
change is one of the few constants in technology, the 
business/academic connections must be revisited 
often"[l]. 

THE PROBLEMS 

In an attempt to remain competitive, companies and 
organizations that rely on computers are being forced to 
continually upgrade their systems. This is not only 
costly, but in a period of time when the emphasis has 
been on corporate downsizing (right sizing), mergers, 
etc., it is extremely difficult to obtain and/or keep 
personnel who are knowledgeable in the use of the latest 
computertechnology. Forexample, "We see a multitude 
of new job titles in this arena that did not exist ten years 
^80 (e.g., database analysts, telecommunications/ 
networking specialists, EDP security/auditing specialists, 



user support analysts, and office automation specialists)" 

[ 2 ]. 

In addition to the problems which are caused by the 
rapidly changing technology, the budgetary constraints 
underwhich m anyMIS Departments are workingmakes 
it very difficu It, if not impossible, to provide the training 
needed to keep personnelup-to-date. On the other hand, 
it has long been considered the responsibility of our 
educational system to provide graduates with the 
background and skills necessary to be successful in their 
chosen fields of employment. For that reason, when 
employers do recruit graduates, they look to colleges and 
universities with a curriculum that utilizes new 
technology and emphasizes current practices [3]. 
Because of the rapidity in which technological changes 
are taking place, computer educators are finding many 
obstaclesto fulfillingthis responsibility. These obstacles 
include limited budgets, lengthy curriculum review 
cycles, obsolescence of hardware, software, proliferation 
of new course topics, obsolescence of faculty skills, and 
the difficulty of recruiting qualified faculty. These 
obstacles are further magnified when you consider the 
ever changing demands of industry. Historically, the 
degree requirements that were in effect at the time a 
student entered college are the same requirements that 
must be met four, five, or even more years laterwhen the 
student is ready to graduate. Although this policy was 
initially designed to protect students who might never 
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graduate due to ever changing requirements, it also 
means that students who are majoring in a technology- 
oriented discipline maybe graduating with skills that are 
obsolete by the time they complete the stated 
requirements for graduation. 

Another major problem faced by colleges and 
universities is a problem similar to that faced by 
industry: keeping instructors up-to-date and qualified to 
teach students how to use the latest computer hardware 
and software. Even if instructors do find ways to keep 
current personally and then attempt to modify course 
content to include the use of new technology, they often 
find it necessary to develop their own teaching m aterials 
because there are no up-to-date instructional materials. 

Based on the experience o f the author, the problem most 
frequently mentioned by educators and students is the 
lack of hardware and software representing the latest 
technology. Some academic institutions are attempting 
to solve this problem by requiring students to provide 
their own personal computers and relatedsoftware. This 
results in a shift of the budgetary problem from the 
institution to the student and, at best, is only a temporary 
solution because the institution is obliged to, but doesn't 
always, provide appropriate hardware and software to 
the faculty for research and course development. 

So far, numerous problems have been identified, but few 
solutions have been offered. In fact, the author believes 
that the above problems will only become worse if 
educational institutions and industries do not work 
together to solve these problems. Organizations need 
employees who are skilled in the use of the latest 
computer technology, but they don't have the time or 
trainers to complete the training. Academic institutions, 
on the other hand, have faculty with the expertise to 
provide the training needed by industry, but are unable 
to acquire the latest versions of hardware and software. 
"Some researchers have concluded that businesses and 
universities share somesimilarchallenges, and increased 
cooperationbetween the two entities will assist in shared 
solutions for both" [4,5], The author believes that the 
key word is "cooperation." By combining their 
resources and partnering together, many of the problems 
and shortcomings faced by each can be alleviated. Based 
on my experience, it is time for universities to join forces 
with industry and use creativepartnerships to the fullest. 
This paper discusses how. 



PARTNERSHIPS TO EXPLORE 

An important way in which corporations may invest in 
the future is by funding university research and 
development efforts in technology - related efforts. 
Donations by corporations maybe in the form of money, 
computer hardware/software, donated and paid employee 
time, efforts, skills and training. Nine ways in which 
partnerships can be formed are summarized in Table 1 
and discussed and illustrated with examples in the 
following sections of this paper. 

TABLE 1 

PARTNERSHIP OPPORTUNITIES BETWEEN 
INDUSTRY AND ACADEMIA 

1 . Advisory Committees 

2. Student Internships 

3. Industry Projects for College Seniors 

4. Faculty Internships/Consulting 

5. Faculty /Industry Exchange Programs 

6. Faculty Training and Updating 

7. Hardware/Software Resource Sharing 

8. Mentoring 

9. Research Grants and Contracts 
Advisory Committees 

A problem that is of particular concern to CIS/MIS 
educators is that of attempting to keep the curriculum 
up-to-date. Which software packages, programming 
languages, systemsmethodologies, and case tools should 
be taught and/or utilized? What hardware should be 
used? More often than not, there is no single answer to 
these questions. Eachcampus must determine the needs 
of the community it serves. That is, what are the needs 
of the major recruiters or companies in the area? The 
intent here is to identify the needs of companies 
collectively and not individually. A campus can’t be all 
things to all people. 

Rather than work in a vacuum, universities should 
determine what computer hardware, software, and 
applications are being used in the organizations which 
hire their graduates. An excellent way to accomplish 
this is through the formation of an advisory committee 
which is made up of alumni, MIS Managers, and 
recruiters. The individuals selected should be in a 
positionto knowthe needs oftheir organizations, as well 
as having a feel for the direction that the computer 
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industry is moving. Traditionally, alumni of the 
program and managers who have hired graduates of the 
IS program are in a position to provide input regarding 
the relevancy of the conte nt of individual courses and the 
curriculum as a whole. 

It is recommended that this advisory committee meet as 
a body at least once each year. Preferably, the meeting 
would be scheduled one or two months before the 
deadline for submitting curriculum change proposals. 
The agenda for each meeting should include a report on 
hardware and software in use or on order by the 
university, curriculum changes being considered, and 
any problems since the last meeting of the committee. 
Faculty requests for resources and curriculum changes 
that have been recom mended o r supportedby an industry 
advisory committee are usually looked upon more 
favorably by the campus administration than those 
submitted by departments that do not utilize advisory 
committees. In addition, it has been the author's 
experience that organizations that have an employee on 
an advisory committee are much more likely to enter 
into some form of “partnership” with the university. An 
advisory committee could be structured as an affiliates 
program or friends of the department where members 
pay a fee to belong (e.g. UCLA affiliates). A word of 
caution, members of an advisory committee must 
understand and accept the fact that they are advisory 
only and not in a position to dictate curriculum and/or 
practices to the university. 

Example: California State Polytechnic University, 
Pomona invites industry input. Although not referred 
to as an advisory committee, California State 
Polytechnic University, Pomona (Cal Poly, Pomona), 
utilizes input from industry in its decision process as it 
relates to curriculum and course content. The faculty of 
the CIS Department at Cal Poly, Pomona traditionally 
hosts a luncheo n for industry “friends of the departm ent” 
approximately once each quarter. About 20 corporate 
recruiters, MIS managers, and key alumni are invited to 
these luncheons with different industry representatives 
being invited each quarter. The faculty of the CIS 
Department make a short presentation after lunch to 
describe what has been accomplished during the past 
year, problems that have been encountered, and changes 
(to hardware, software, and curriculum) that are being 
considered. Each attendee is then given the opportunity 
to describe changes in MIS being considered and/or 
implemented by his/her organization and to provide 
comments, both positive and negative, about the changes 
being considered by the department. The input from the 



corporate representatives has proven to be extremely 
valuable in the final decision process at Cal Poly, 
Pomona. 

Example: PeopleSoft establishes alumni outreach 
program and academic advisory council. An emerging 
model of partnerships is heavy involvement of alumni 
who work in industry to help with the recruitment of 
new graduates. PeopleS oft, a software company based in 
Pleasanton, CA, is successful at recruiting large numbers 
oftechnicalpersonnel through alumni. The firm usually 
targets its recruiting efforts at about 10 universities 
annually, for example, Carnegie Melon, Cal Tech 
(Pasadena), Howard and Tuskegee Universities. The 
alumni of these schools who are team leaders make 
frequent visits to these campuses over the course of a 
year. Occasionally informational speeches are made to 
technology students to inform and attract them to 
Peoples oft. 

The "Peoples oft on Cam pus" initiative is geared toward 
integration of PeopleSoft software training into the 
curricula of colleges. Developed by PeopleSoft's 
Academic Advisory Council to meet needs of the 
academic community, the program centers on how the 
business world utilizes PeopleSoft. Beginning in 1999, 
PeopleSoft Human Resource Management and Financial 
Software (HRMS) has been distributed free of chaise to 
Universities for students to experience first hand. As 
part of this pilot program, PeopleSoft representatives 
came directly to Cal State, Hayward in support of this 
program. It is appropriate to comment here that the role 
of the PeopleSoft Academ ic Adviso ry Coun cil has more 
of a corporate orientation than most advisory boards for 
universities. 

Student Internships 

The term internship is used to describe all programs 
where students work part-time in a position related to 
their chosen career field while continuing to make 
progress toward their degree objective. Student 
internshipsare not new, but they are often not utilized by 
industry. The question might be, why aren’t student 
internships used more by businesses considering the 
many benefits thatcan be derived? Some of thebenefits 
to the employing agency are: (1) a ready source of 
"skilled" applicants, (2) an opportunity to test employees 
without a long-term commitment, (3) a relatively 
inexpensive source of labor, (4) highly motivated 
employees, and in some cases, (5) applicants who have 
been "prescreened" by the university. According to a 
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1994 New York Times article, ’’Employers nationwide 
are hiring four of every five of their college interns full- 
time after graduation” [6]. 

The benefits are not limited to employers. Internships 
also provide benefits to the university. An internship 
provides (1) a real world test of the academic program, 
(2) a supplement to classroom learning, (3) access by the 
student to expe nsive re source s, (4) a ready market for the 
product (graduate) of the university, and (5) a motivated 
student because classroom learning becomes more 
relevant. 

Example: California State Polytechnic University, 
Pomona has a well-established internship program with 
approximately 65-70 students receivingacademic credit 
for their work experience each quarter. Because some 
organizations may have a policy (sometimes union 
controlled)against hiring part-time employees, Cal Poly, 
Pomona has established contracts to provide “temporary 
MIS employees” with two major organizations. The 
most successful program has been in existence with the 
County of Los Angeles for over ten years. In that 
situation, the university fills temporary positions with 
qualified students and, in turn, bills the County for the 
students’ services. A pay scale has been agreed upon as 
part of the contract whereby the pay is different for 
lower-division, upper-division, and graduate students. 
In fact, the contract also specifies an hourly wage for 
faculty consultants. The agreed upon pay scale includes 
an “administrativecharge” for administeringthe payroll 
function for the contract employees [7]. 

Industry Projects for College Seniors 

Complex subjects are learned best by students if the 
students are involved in a variety of different teaching 
methods. Students can read about the subject, observe 
demonstrations, listen to lectures by an instructor, or be 
actively involved in the learning process through the 
assignment of a project or case study. Students usually 
learn best by physically doing something. In this case, 
the more realisticthe project or case study assignment is, 
the more valuable the learning. 

Since most companies have a backlog of systems and 
programming projects, it would seem quite likely that 
one or more of those projects could provide a valuable 
learning experience to students. A key consideration in 
selecting real projects from industry is that the projects 
must: (1) fit the timeline of the term, either 10 or 16 
weeks, (2) have a required deliverable at the completion 



of the project, and (3) provide a learning experience 
consistent with the learning objectives ofthe course and 
curriculum. 

Real-life senior projects from industry provide a good 
introduction to the problems that will be faced after 
graduation. Of particular value is the students’ 
realization that communication skills are important if 
they must work in a team environment and meet 
periodically with the industry liaison. The university 
benefits inasmuch as many of the projects involve 
hardware and/or software that is not available at the 
institution. The cooperating organization gains by 
having a low priority project completed at virtually no 
cost. Based on the experience ofthe author, companies 
that do end up implementing the student-developed 
results are often quite willing to make a contribution 
(hardware, software or money) to the university. 

Example: California State University, Hayward 

arranged for a team of seniors in CIS to work on a 
database design project for Oracle Corporation. After 
graduation the students were offered employment- 
evidence of a mutually beneficial arrangement. 
Subsequently, Oracle do nated software to the University 
for use in student labs. 

Example: California State Polytechnic University, 
Pomona utilizes real-life projects in its capstone course 
for all CIS majors, CIS 466 Systems Development 
Project. Student teams of 4-5 students are assigned to 
each project which culminates with a formal written and 
oral presentation (defense) of the project solution to the 
management group which provided the project. 

Faculty Internships/Consulting 

Faculty internships, unlike student internships, usually 
involve full-time employment during the summer. In 
some instances, a faculty internship could require the 
faculty mem ber to take a n unpaid leave of absence from 
teaching for one or more terms. Regardless of the 
duration of the internship, the employer and university 
both enjoy benefits from the program. The employer 
gainsfrom the expertise ofthe faculty member, while the 
faculty member gains insight into the use and application 
of some of the latest computer technology. 

Example: IBM occasionally hires a college professor 
on unpaid or sabbatical leave. IBM also participates 
in employee exchanges that include University faculty as 
well as IBM employees. For example, IBM formed a 



100 



Proceedings of the 15”’ Annual Conference of the International Academy for Information Management 



cooperative program with the University of North 
Carolina at Chapel Hill to educate faculty from around 
the United States to learn new research and instructional 
uses of computer technology. The main purpose o f this 
program is to integrate technology into the curriculum 
by using computer equipment and packaged software. 

Faculty/Industry Exchange Programs 

A few large corporations, such as International Business 
Machines, have been involved in faculty/ industry 
exchange programs for many years. In this program, a 
faculty member is assigned to work for the cooperating 
organization while an employee of that organization is 
scheduled to work at the university. That work might 
consist of teaching courses, assisting in new course 
development, and/or training other faculty in the use new 
hardware or software. These ex changes are usua lly of a 
short duration, usually one year or less, and the salary 
and fringe benefits of thepeople involved are maintained 
by the original employing agency. 

Example: IBM has a faculty loan program. The 
faculty loan program is an agreement between senior 
IBM employees specifically located in North America 
and the university of the employee’s choice. A 
maximum of 7 employees per year of some 200,000 in 
North America are selected to participate in this 
program. Once an employee is selected, s/heis placedin 
a "leave status” from their currentposition within IBM 
to become an acting university faculty member at an 
American University. 

Typically the recipient Universities are delighted to 
obtain a "free” professor for any amount oftime. Part of 
this time is used to plan for these programs by IBM and 
the participating universities before the partnership 
agreement is complete. The legally binding one-page 
Document of Understanding (D OU) is prepared to avoid 
any legalproblems.The appointment is arranged initially 
for one year but can be extended upto athree-year term. 
The employee assumes the role of faculty member in 
his/herarea of expertise in the appropriate department of 
the university. 

Participating universities do not compensate the acting 
professor and are not obligated to IBM through any 
financial, personnel, or informational exchanges. The 
employee remains on the IBM payroll and is reimbursed 
for all expenses related to the assignment to the 
university. IBM management perceives the costs 
incurred as a long-term investment in the technological 



education of US students. Upon completion of the 
assignment, the employee may return to the pastposition 
at IBM or venture into a vacancy at his/her management 
level. Sometimes the employee may opt to retire from 
IBM and join the University feculty for an extended 
period oftime if not permanently. 

Faculty Training/Updating 

One of the problems faced by both businesses and 
academic institutions is that of attempting to keep 
personnel up-to-date with the latest technology. Most 
large organizations attempt to overcome this problem by 
providing training for their employees. In many 
instances, this involves paying all expenses to send one 
or more employees to another training site for a few 
days--in some cases one or more weeks. This is 
extremely expensive if many employees must be 
provided 'with the same training. In those cases, 
organizations often hire a professional trainer and hold 
the training sessions at the local place of business. 
When this is done, the cost of the training is usually a 
fixed amount regardless of the number of individuals 
being trained. Therefore, it costs the company nothing 
to invite one or two faculty from the local university to 
attend the training session. The only cost to the 
university is that of providing a substitute teacher for the 
classes missed and, in many CIS/MIS Departments, the 
missed classes are taught as an overload by other faculty 
in the department as is traditionally done at Cal Poly, 
Pomona. 

Hardware/Software Resource Sharing 

In the past, many companies have donated computer 
equipment that they no longer need to academic 
institutions. Although this is still a possibility, there is 
a high probability that any equipment being offered as a 
donation is already obsolete and will notbe of any major 
benefit to an acade mic institution either. Another way in 
which access to needed hardware/software can be 
provided to universities by industries is via networks. 
Since most companies are now using networks to 
communicate with different entities within their 
organization, the CIS/M IS Department at the university 
can be established as a node on the company’s network. 
It may be necessary to limit access by the university to 
certain hours of the day so that the ongoing operation of 
the company is not impacted. In addition, care must be 
taken by the company to assure that access is controlled 
and limited to only those areas previously agreed upon. 
With proper controls, valuable resources can be shared 
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with an academic institution with virtually no extra cost 
to the company. 

Some much needed software cannot be obtained by 
universities because of the high cost of licensing. On the 
other hand, it is quite possible that a localcompany may 
have a licensing arrangement that provides more copies 
of software than that company actually needs. For 
example, a company that needs 30 copies of a software 
package may find that the cost is the same for 25-50 
copies. Some of the excess copies, with permission of 
the software company, can be "loaned" to the university. 
In the long run, the company benefits because graduates 
will be trained on thesoftware that companiesare using. 

As noted above, the sharing of software or use of 
software licensedto another person or entity should on ly 
be done with the written permission of the software 
vendor. Based on the author’s experience, such 
permission can usually be obtained if the software 
vendor is involved early in the decision process and 
knowshow and where the software will be utilized in the 
curriculum. 

Example: Microsoft software donations. Of the 3.4 
million estimated technology jobs nationwide, 10% are 
vacant, thereby slowing the growth of technology-based 
industries. (Virginia Tech University Study, 1998). By 
donating software to high schools and colleges, 
Microsoft hopes to persuade students to select a 
technical professional career and eventually relieve the 
shortage of technically qualified workers. Furthermore, 
investing in development of future technical 
professionals is consistent with the Microsoft goal of 
retaining 90% market share. 

Microsoft launched a software donation project to 
benefit middle schools, high schools, vocational and 
technical schools, and universities that specialize in 
computer science and technology training. In 1998 
recipient schools were given over 18,000 software 
licenses, Microsoft Visual Development tools and 
operating systems. Software licenses were for Visual 
Basic, Java Professional Edition, Visual InterDev Web 
development. Office 97 Developer Edition, and 
Windows, or NT operating systems. Through these 
software donations Microsoft helps to provide practical 
solutions to complex challenges facing high schools, and 
especially college and university computer science, 
engineering and information systems departments 
attempting to integrate new technology into their 



curriculum. A web site was prepared to assist faculty in 
the integration of these tools into the curriculum 
(www.m icrosoft.com). 

Example: Intel partnerships. Intel began a three-year 
partnership grant program called Education 2000 for the 
advancement of information technology research and 
development and curriculum development. To win a 
grant, universities must demonstrate academic 
excellence, commitment of the university to support the 
grantobjectives,andthepotential benefitto students and 
faculty. Intel takes the initiative by sending a team of 
people to the universities identified, interviews the 
faculty, and determines what the needsare and how best 
to help the university. The donated equipment helps to 
keep the university computing facilities on the leading 
edge of technology. The main disciplines to benefit 
from the Intel grants so far are computer science and 
engineering. Additional disciplines that may become 
eligible include anthropology, astrophysics, medicine, 
biotechnology, business, public policy, journalism and 
the arts. Grants are made in the form of high-speed 
multi-media computers, workstations, servers, and 
networking hardware and software. 

Not only is software donated, but funding is provided for 
various projects such as developing next generation, 
content based networked multi-media technology for 
"digital storytelling." Recipients have the option of also 
working on specific needs for their schools. For 
example, Columbia University, has optedto upgrade all 
digital operations. As a direct result of an Intel grant, 
Columbia upgraded the entire infrastructure of its 
journalism school which conducts collaborative research 
with the engineering school by using Intel equipment to 
investigate multi-media searching and storytelling 
applications. 

Intel and other companies with similar programs often 
find it easier to work with schools and departments that 
have established industrial relations departments to 
interface with industry (www.Intel.com). 

Schools located in geographic proximity to Intel and 
otherlarge companies standto benefit from partnerships 
also. By seeking to improve the curriculum, computing, 
and software environment of local schools, these 
multinational corporations hope ultimately to hire local 
graduates, which saves the firm relocation expenses 
upon hiring. 
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Mentoring 

Mentoring programs usually focus on attracting a 
diversity of students to technical majors such as 
engineering, computer science, information systems and 
the sciences. Either a representative from industry, a 
faculty member or graduate student will team up with 
and assist female and minority students in particular 
who may have difficulty with prerequisite subjects such 
as math. Successful mentoring programs lead to the 
selection of technical majors by these students and 
facilitate their successful completion and eventual 
employment. The company that sponsors the mentor 
benefits by increasing the pool of potential employees 
and the individual student benefits by enhancing their 
employability through improved education. 

Example: Motorola establishes mentorship 

programs. Business models have evolved over the past 
several years to incorporate faculty and university 
administrators into the "learning leadership team model 
and initiative" headed by Motorola, beginning in 1992. 
The main focus of these models is for faculty to 
understand that all changes in reference to technology 
education must be systematic, with ateam to support this 
change and that a stud ent-focuse d approa ch mus t be used 
to determine the specific areas for future change. 
Northeastern Illinois University has been the academic 
leader for these partnerships, including the Minority 
Mentorship Program, which matched incoming freshmen 
with faculty who help to build a personal support 
relationship, www.motorola.com) 

Example: Hewlett Packard partners to mentor 

women, minorities. Hewlett Packard has been apioneer 
in a variety of partnerships to further educational 
enhancement within the technology arena. In 1997 HP 
announced a partnership to broaden opportunities for 
women and minorities in the engineering, computer 
science and related technical fields. Called "Diversity in 
Education Initiative," HP entered into partnerships with 
various universities and school districts to focus on K 
through 12 levels of education. This program provides 
monetary gifts to universities that work together to 
prepare females and minorities for college level math 
and science courses as a foundation for pursuing 
engineering and computer science majors in college. 
This program targets majors where women and 
minorities are under represented, due to difficulties in 
math and science. 



HP provides scholarships to high school and college 
students and provides job opportunities for them while 
attendingschool. These support systems are provided to 
increase the graduation and employment rates ofwomen 
and minorities specifically. HP management felt that 
this extra effort of philanthropy would spark interest 
among other organizations to follow suit. 

Participants involved in this program are the University 
ofCalifornia,Los Angeles with the Los AngelesUnified 
School District, San Jose S tate University , working with 
Alum Rock Elementary School and East Side Union 
High School Districts, Northeastern University, Boston 
with Boston Public School district, Lower Roxbury area, 
and the University ofTexas at El Paso with the El Paso 
Independent School District These universities 
collaborate with each other to work with several high 
schools, middle schools and elementary schools. 

All participating Universities are on a five-year plan to 
implement math and science enrichment and mentoring 
programs for high-school students to increase retention 
rates of female and minority students with a specific 
focus on preparing for computer science, A total of 
$ 1 25,000 has been received for each of the five years; 
each school annually receives $25,000 to support these 
initiatives. Partners within the K-12 areas receive 
$290,000 each to carry out math and science programs 
on each grade level. Engineering and Computer Science 
students have the opportunity to apply for $3000 
scholarships provided by HP to each of the university 
partners for this project. 

The ultimate goal isto alleviate the statistics that women 
and minorities drop out of engineering and computer 
science majors more often than Caucasian males. 
Hewlett Packard hopes to increase the mathematical 
skills so that these students are well prepared to enter 
into technology specific industries and compete well in 
college. HP has also encouraged universities and other 
corporations to offer technical classes for students. 

In 1996 Hewlett Packard continued its support forthese 
and related programs by donating over $40 million in 
cash and products to improve the quality of education 
from kindergarten to the university level. The Univer- 
sity of Washington and the California State University 
System are among the many beneficiaries of Hewlett 
Packard donations (www.hp.com). 



Proceedings of the 15“' Annual Conference of the International Academy for Information Management 



103 



113 



Research Grants and Contracts 

Many corporations see funding university research and 
development efforts in the area of technology and 
science as a valuable way to invest in the future. 
Donations may be in the form of money, computer 
hardware/software, or even employee time. Relation- 
ships with industry have eased the financial burden of 
many universities which in the past were not able to 
conduct intense research due to the limited availability 
of governme nt funding. University rese arch has become 
increasingly dependent on corporate donations. 
Corporations have created and opened the window for 
outreach and technical growth while in conjunction these 
donations provide students with the chance to continue 
their knowledge with hands-on experience using current 
technology in industry. Universities are buildingoppor- 
tunities for faculty and staff that will provide the 
knowledge needed to help students quickly acclimate 
into the work force with computer literacy, knowledge 
about software applications, programming and operating 
systems. The corporations benefit by gaining access to 
the best students for potential hire and to innovations 
that lead to new products. Thus, collaboration between 
faculty, principal investigators, national laboratories, 
non-profit research centers and industry researchers can 
be mutually beneficial. 

Example: Sun Microsystems facilitatescollaborative 
research. Sun Microsystems, located in the Silicon 
Valley of California, has been actively involved in the 
educational community specific to technology. 
Collaborating with faculty, research directors and 
principal investigators at universities, national 
laboratories and nonprofit researc h organ ization s, Sun’s 
presence is very large in this geographical area. 
Consistent interest and interaction with these 
organizations through collaborative efforts have helped 
to enhance the success and future of Sun. All projects 
that Sun participates in have the common mission, 
’’Future technology having commercial importance 
within the next two .to five years ultimately providing 
direction for engineers and management at Sun” 
(www.sun .com/products-n -solutions.edu). 

Sun has founded the program and created provisions for 
easy accessibility for new projects. Graduate level or 
undergraduate level students may simply discuss 
partnership ideas with faculty advisors, who in turn 
communicate these ideas to Sun. The mission of Sun 
collaborative research is to recruit and hire the best 
students, encourage and promote faculty research and 



sabbaticals, student internships, transform university 
research into new Sun products, and to create future 
trends in computer and information technology. The 
initiative owner, along with the Sun Technical Sponsor, 
oversees the implementation of the particular project 
(www.sun.com/products-n-solutions/edu). 

Example: Apple Computer partners with Carnegie 
Mellon. Apple Computer joined forces with Carnegie 
Mellon University in a collective effort of resources to 
build a Mac II version of Mach, a multiprocessor 
operatingsystem. Over a two-year periodof production, 
the final product, Mac IIs, is available to run Project 
Andrew application. Project Andrew is a wide area 
network that allows users to run applications on the 
networks of other machines (www.apple.com). 

STANFORD UNIVERSITY: 

PARTNERSHIP PIONEER 

Stanford University, known for bridging and building 
many technologically based partnerships with 
corporations globally, reported that in 1998 the 
University was actively involved with 20 corporate 
partners. With the consistent donations ofthese corpora- 
tions, Stanford hasbeen able to explore and expand co re 
research, enhance outreach capabilities, and enter into 
new collaborative research projects. These partnerships 
have enhanced the curricula of several S tanford sch ools 
and departments and have increased awareness of the 
importance of information technology and input from 
public policy makers on technology-specific topics 
(www.stanfo rd.edu/group/scip/). 

Stanford computer industry project for software 
research. Using the funding received from many of its 
partners, Stanford University has entered into a strategic 
research project specific to software. Another venture 
resulted in Stanford conducting research about 
segmentation in the software industry. The focus of this 
study, conducted under the auspices of the Sloan 
Foundation, is current patent law, piracy, trade policy, 
labor, immigration, education, antitmst, international 
competition, quality and technology change (www.- 
scip.stanford.edu/scip/). 

Strategic uses of information technology project 
(SUIT) facilitated by education forum. Stanford 
created an educational forum in 1996 geared toward 
senior level executives to meet and discuss technology 
needs of the business world. After much success in the 
first forum, the project was incorporated into the 
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Stanford graduate-level executive education program. 
Various portions of th is project have been presented and 
well-received at conferences around the world. Thus, 
the educational forum assumed the role of an advisory 
board to help Stanford get started with SUIT 
(www.stanfo rd.edu/group/scip/). 

SUMMARY 

The above suggestions are only some of the ways in 
which organizations and universities can partner to 
overcome some of their problems and achieve their 
objectives. It should be noted that the benefits that can 
be attained farexceed any costs thatmay be involved in 
the implementation of these suggestions. Many of the 
suggested areas for partnerships presented in this paper 
have been proven effective at universities where the 
author has taught. The author is of the opinion that all of 
the suggestions for partnership ventures could work to 
some degree for all colleges and universities. 
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EDUCATION AND TRAINING IN MIS 
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Georgia Southern University 



PANEL PROPOSAL 



This panel will explore the differences between 
education and training in the context of MIS curriculum 
development. The discussion will consist of examining 
the relevant premises for comparison and the need to 
understand the pedagogical differences. Relevant 
premises are based on various research studies (Cohen & 
Lippert, 2000; Lippert, Granger, & Lydon, 1 999; Lippert 
& Granger, 1998) coupled with panelists’ experiences, 
observations, working papers (Cohen & Lippert, 1 999b), 
publishedmanuscripts (Lippert, 2000; Cohen & Lippert, 
1999a), and presentations (Cohen & Lippert, 1998a, 
1998b). 

A role of academic MIS educators is to provide a broad 
theoretical foundation grounded in MIS principles and 
procedures for aspiring technology and business 
professionals. Educatio n, operation alized as, the gradual 
process of acquiring awareness through instruction and 
activities which enlighten and impart knowledge, 
furnishes this theoretical foundation for life-long 
learning. Education provides breadth and scope of 



instruction with high levels ofgeneralizability to similar 
situations. 

In a field of study, where change is constant and 
technologies frequently modify, both internal and 
externalpressuresmay challenge MIS educators to offer 
instruction on the utilization and manipulation of the 
latest technologies. This instructional approach where 
instruction is specialized and aimed at skill transference 
and applicability to a specific task or technology is 
operationalized as training. A goal of technological 
training is proficiency at performing a specific task. 

Theoretical, practical, and pragmatic differences exist 
between education and training in terms of admission 
criteria, purpose of instruction, learning objectives, 
instructional procedures, scope of instruction, instruction 
timeframe differences, delivery methods utilized, 
instructor knowledge and experience based on terminal 
degree status, and expected outcomes. 
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Contextual differences between training and education 
exist with respectto business decision-makingprocesses. 
Training addresses fixing the problem where education 
addresses diagnosing the problem. The desired end state 
is technological usability relevant to managers in their 
business decision-making process. 

Part of the student’s educational experience is interest 
cultivation in exploring and answering unanswered 
questions. In addition, part of the educational process is 
to teach students how to think and how to deduce 
relevant facts from previous learning. Training is 
instruction presented with emphasis placed on 
accomplishing the instantiation of the task and may not 
offer the needed basis for future growth and intellectual 
connections. 

Yet, the often-expressed polarity between these two 
learning approaches may foster an either/or mindset 
possibly excluding integrated instructional strategies. 
MIS curriculum alternatives spanning the polarity divide 
include: 

• A theory based course devoid of hands-on 
laboratory instruction 

• A skills development course or 

• A unified bi-component course comprised of a 
theory-based element coupled with an integrated 
hands-on laboratory component. 

However, from an educationalperspective,a blending of 
theory and the hands-on experience may offer students 
theoretical grounding and practical application within a 
guided learning environment. For example, educators 
may develop real world exercises with Oracle’s Designer 
2000 for data modeling or Developer 2000 to provide 
hands-on experience with database concepts. Novell 5 
may be used to reinforce Network Operating Systems 
concepts. If students can be ‘educated’ while being 
partially trained on ‘hot’ technologies, shouldn’t we as 
educators be doing so? 

ACM curricula recommendations offered as the IS’97 
Model Curriculum advocate thatall Information Systems 
students possess computer literacy in skills usage and 
application (Davis, et al., 1997, 9). The skills 
prerequisite is typically acquired before formal courses. 
The six types of applications (Internet and electronic 
mail, spreadsheet processing, database management. 



presentation graphics, statistical analysis, and external 
database retrieval) form the competency base for all IS 
students (Davis, et al., 1997, 10). IS’2000 

(http://cis.bentley.edu/isa/pages/ is2000.html) is the 
update of IS’97. 

These contextual differences between training and 
education in MIS curriculum development are further 
illustrated when 

• The issue of course credit is discussed 

• The potential absence of theoretical underpinnings 
related to specific package usage is considered 

• The importance of marketable stills obtained by 
graduates is stressed 

• Different student learning expectationsare evaluated 

• Different student interests are contemplated and 

• The increasingly diverse skill ranges of student 
abilities entering the university are addressed. 

Intellectual exchange stimulated through an environment 
of open discussion is proposed for this panel 
presentation. 
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COMPUTER APPLICATIONS COURSE 
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Extended Abstract 

It is generally accepted that change is the only constant. If that is correct, it should be anticipated that students 
entering a university-level introductory computer course bring differentcomputer experiences today than previously. 
That was the premise which prompted a change in core requirements for most School of Business majors at this 
university. Students are now required to complete an advanced microcomputer course rather than the beginning 
course. This study was undertaken to assess the computer skills of students entering the introductory course at this 
university. Do our students possess improved and broadened computer skills in the use of the software applications 
being taught in the introductory course? Are most students are ready for an advanced microcomputer applications 
class rather than the introductory class? The project is ongoing; however, this paper discusses the study and 
preliminary results. 



BACKGROUND 

The introductory level computer ^plications course has, 
at many universities, undergone a series of changes 
during the past 10-15 years. At the university in which 
this study was conducted, those changes have been, 
primarily, technology driven. This study was 
undertaken to determine whether those changes, and 
consequently the current course content, meet student 
needs. 

Though initially a class taught solely by an instmctor, 
for several years the introductory computer course at 
this university has been taught via computer-aided 
instruction (C AI) with minimal faculty oversight. In the 
last two or three years, this course has also been taught 
as a distance education course. 

Technology has driven not only the method of delivery 
for the introductory computer course but also the course 
content. It has catapulted the course through 
programming, DOS, Windows, and various software 
applications. In a very short time span, application 



packages increased in capability and complexity, and 
advanced from DOS- to Windows -based, with an 
accompanying increase in time needed to adequately 
cover a fair number of components. The software 
applications taught in the introductory course also 
changed with technology as integrated packages were 
touted as advantageous and as Bill Gates’ marketing 
strategy to bundle software and hardware took its toll in 
the market and in the classroom 

RATIONAL 

The content of a university course should be a function 
of at least two considerations: employer expectations 
upon graduation and student preparedness at entry. 
Other probable considerations include facilities, 
equipment, teaching staff, accreditation standards, and 
additional course offerings. The introductory computer 
applications course at this university has evolved from 
a course for CIS maj ors through a course for all business 
majors into a university service course. The course 
content is now an introduction to Windows, word 
processing, spreadsheet, database, and presentation 
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software. It is also the prerequisite for the advanced 
microcomputer applications course. 

Recent curriculum changes in our School of Business 
sparked much discussion about the introductory-level 
computer course vs. the advanced microcomputer 
applications course, the content of each course, their 
purposes, and their placement in the curriculum. As 
high schools increasingly incorporate a similarclass into 
their curriculum, entaing university freshmen, some 
argue, should already possess introductory-level 
computer skills. Indeed, an increasing number of 
students are "testing out" of the introductory computer 
applications course. Further, almost every major within 
the School of Business has recently changed its core 
requirement to the advanced microcomputer 
applications course, not the introductory course. Should 
the introductory computer applications course, then, be 
considered remedial? Have most students already 
mastered the skills taught in the introductory course? Is 
its content appropriate for its clientele? Does it serve 
the preparatory role it should for the advanced class? 
Perhaps, just as a minimum level of English and math 
competency is expected, the time has come to expect a 
minimum level of computer skills upon college 
entrance. 

HYPOTHESES AND METHODOLOGY 

The hypotheses of the study were that students entering 
the introductory-level computer course: 1) have had 
prior experience with word processing, spreadsheet, 
database, and presentation software; 2) have used 
email; 3) have used the Internet for entertainment but 
not as an educational or business tool; and 4) own a 
computer. 

To obtain information to develop a profile of students 
entering the introductory computer applications course, 
an instrument was developed and administered to 85 
students in the on-campus classes during the Spring 
semester of 2000. Of those, 72 usable surveys were 
obtained. The survey will be administered again during 
the Summer sessions of 2000 (currently 60 students are 
enrolled) and Fall 2000 (probably, 200+ students). 

The instrument was constructed to obtain information 
concerning the type of computer(s) previously used, the 
nature and location of any experience indicated, access 
to a computer away from the university, and 
demographic data. 



Previous computer usage was addressed by questions 
concerning the time spent using a PC and/or a mini or 
mainframe computer and the classification of any 
computer(s) personally owned. The nature of the 
respondent’s computer experience was assessed by 
questions concerning the amount of time spent using 
email and the Internet as well as questions concerning 
proficiency with specific software applications. The 
location of the respondent’s experience was obtained 
from questions concerning the number of semesters of 
course work completed in an educational environment 
and on-the-job training. 

PRELIMINARY FINDINGS 

This study nor its analysis is complde. Preliminary 
results, however, do point to some interesting findings. 
The first hypothesis was that students entering the 
introductory-level computer course have had prior 
experience with word processing, spreadsheet, database, 
and presentation software. Word processing and 
spreadsheets were the only software applications with 
which a majority of the students claimed to have a 
degree of proficiency, and 10% claimed no prior 
computer experience at all. 

The second hypothesis was that students entering the 
introductory-level computer course have used email; yet 
58 percent of the students reported no email usage. 
Another hypothesis was that students entering the 
introductory-level computer course have used the 
Internet for entertainmait but not as an educational or 
business tool. A surprising 63% of respondents 
indicated no Internet usage. Of the 40% who did use the 
Internet, almost half was for pleasure. That leaves only 
20% of our students using the Internet as an educational 
or business tool. The final hypothesis was that studaits 
own a computer, and indeed, 75% of all respondaits 
indicated that they owned a computer. 

As computers continue to penetrate the fibers of life, 
educators strive to keep pace. Since the field of 
computers is probably the most radically changing of all 
fields, constant new research and frequent repetitions of 
research are necessary to evaluate the correctness of 
current and future educational goals. This endeavor is 
but one step toward "keeping pace." 
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ABSTRACT 

Asynchronous learning (ASL) tools are being used at colleges and universities in many countries around the world, 
but the majority of implementations are to enhance or support the traditional classroom environment rather than to 
provide asynchronous on-line classes. This paper presents a succinct review of some major issues that have caused 
this situation. It then looks at four ways that a particular ASL tool, Blackboard, is used at one school to support 
traditional lecture courses. The paper looks at several qu antitative measures that can be used to determine if the use 
of the ASL tool has any significant effect on learning and the learning environment. The results of a study that 
analyzed forty-three Blackboard supported courses are presented. It describes a logical methodology to evaluate 
changes in student performance in a quantitative manner. The study notes thathigh use of the ASL tool in supporting 
the class consistently results in an improvement in the learning environment based on significant changes in two or 
more quantitative measures. The study also presents a brief review of qualitative comments that highlight student 
concerns about the use of ASL tools in traditional classes. The paper ends with a discussion of some important issues 
for those planning to use ASL tools to augment or support traditional courses and areas for further research and 
study. 



INTRODUCTION 

Web based asynchronous learning is an area of rapid 
growth. In a period of three years from 1995-1998 the 
number of schools in the United States interested in 
developing such programs increased by almost 200 
percent (Morse, Glover, and Travis, 1997, Phillips- 
Vicky, 1998.) There is much controversy about the 
efficacy of distance education in higher education. 
Comments are varied and no one seems to know what 
the final impact of distance education will have on 
traditional programs. Abernathy (1998), Binde (1998), 
Farrington (1999), Smith (1998), and Theakson (1999) 
all discuss some of the current issues involving the use 
of technology in distance education and its potential to 
revolutionize education. The impact o f the programs is 
unclear. Carnevale (2000) highlights faculty issues in 
distance learning, and Blumenstyk (1999) and Daniels 
and Rubin (1998) focus on some shortcomings and how 
several programs have not been successful. Poley 



(1998) describes some specific examples of distance 
learningin American universities and schools around the 
world. Wilson and Meadows (1998) discuss issues in 
distanceeducation for schools in Australia who often use 
organizationsin Asia to assist in delivering international 
information-age education. 

Several companies are producin g software packages that 
provide an environment for faculty to develop courses 
that can then be presented in an asynchronous distance 
learning format. Most of these products allow students 
to use a web browser to view and use course materials 
with no special software required, and they vary widely 
on the investment in time, money, and computer assets 
for implementation. All ASL tools place some 
restrictions on what material can be presented, how it 
can be presented, and also the types of interaction 
provided. Considering the limitations exhibited by most 
of these products, many schools are using them more to 
augment traditional courses rather than usingthem solely 
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to provide stand alone distance education courses. This 
paper looks at how one ASL tool, Blackboard, is being 
employed in a variety of courses at one university. It 
then looks at four measures to detemiine if there is any 
quantitative difference in course outcomes when courses 
that use Blackbo ard to augme nt classes are compared to 
courses that didn’t use the tool. 

BACKGROUND 

The use of ASL started in the 1980s, when some faculty 
and students began to teach and learn asynchronously 
using e-mail (McM ullen, Goldbaum, W olffe, & Sattler, 
1998). As the Internet and browser technologies have 
improved, there has been a major move by schools and 
vendors to utilize this medium in a variety of ways. The 
limitations of these systems are those inherent in using 
technology as a means of presenting education such as 
restricted instructional methods, limited means of 
communication and feedback. Speed of transmission is 
also an important issue as it determines just how much 
or how little information can be effectively received by 
students. 

Three recent studies (Rogers and Laws, 1997, Coyle et 
al., 1998, Novitzki, 2000) identified several 
characteristics and qualities which must be presentin an 
academically sound on-line asynchronous course. The 
three most critical factors are: It should be a self- 
contained standalone package that is convenient and 
easy to use and which depends on nothing else. Second, 
it should provide as far as possible the same educational 
experience as the traditional class. Third, it should 
provide for easy interaction with the instructor and other 
students. No current ASL tool fully satisfies all of these 
requirements, and as a result few schools have been 
successful in putting large numbers of on-line courses or 
whole degree programs on the Internet without extensive 
vendor support and assistance. At a recent meeting 
sponsored by Blackboard (Blackboard, 2000), it was 
reported that the ratio of courses used to augment 
traditional courses to on-line courses was 15:1, fifteen 
augmented courses for every full on-line course. In spite 
of this largepreponderance of augmented courses to on- 
line courses, little has been reported quantitatively on the 
impact that ASL technology in this format has on 
students, courses, or instructors. 

In 1997-98 our school, whose focus is providing 
graduate business education to working adults on a part 
time basis, conducted an analysis of the various ASL 
tools available to develop and offer courses in an 
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asynchronous on-line environment. There are a large 
number of ASL tool vendors who vigorously market and 
promote their wares (Abernathy, 1998, Blumenstyk, 
1999, Frederickson, 1999, Hiltz and Welman, 1997). 
Several of these vendors made presentations and, based 
on our needs and their capabilities. Blackboard was 
chosen as our standard ASL tool for on-line course 
development and support (Novitzki, 1999). 

CURRENT STUDY 

The use of Blackboard in courses has varied across the 
departments at our school. Development of the ASL 
supported courses has been spreading slowly due to 
limited assets and technical support. When faculty and 
staff were learning the tool, only a few instructors could 
be involved. They began slowly, initially starting atlow 
use levels, and then moving to higher levels of use in 
later terms. In the beginning the Information 
Technology (IT) department was the main Blackboard 
user, but later, as assets permitted, other departments 
have been getting more involved in the technology. 
Table 1 clearly shows how other departments have 
become more involved in the use of Blackboard, Its use 
has evolved into four different formats, little use, 
moderate use, extensive use, and on-line courses. Low 
use consisted of putting up a detailed web page that 
described course goals, objectives, and requirements and 
which provided the syllabus and assignments. No 
lecture materials were provided, nor was any testing. 
Assignments were posted as part of syllabus, but they 
were not augmented or supported by the ASL tool. 
There was no option provided for students to 
communicate with each other through the Blackboard 
software. 

Moderate use started with the same web page described 
above, but lecture notes, slides, and other supporting 
material were also available on Blackboard, Instructors 
often added links to other information sources and 
quizzes. If students communicated with the instructor 
outside of class, it was through the use of regu lar e-mail, 
notthroughBlackboard, Normal lectures were held each 
week and the material available through Blackboard was 
designed to augment and explain the material in the 
lectures as well as repeat key material covered in class. 
Material on Blackboard was referred to during classes, 
and students were encouraged to review and use the on- 
line material 

Extensive use involved all of the features described 
above, but also required assignments to be submitted on- 
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line using features found in Blackboard. Additionally, 
these courses used the various communication features 
of Blackboard. Bulletin boards, threaded discussions, 
and chat rooms are all used to augment the class lectures 
and materials. All class materials were submitted via e* 
mail and graded by the instructor similar to that 
described in Braught et al (1998). 

On-line classes have operated generally as extensions of 
the extensive use format. The differences being that 
students do not atten d traditional class lectures, and most 
instructors provide some sort of audio lecture materials. 
The use and effectiveness of these courses is beyond the 
scope of this paper and the on-line courses will not be 
discussed further. 

RESEARCH QUESTION 

From an educational standpoint a key question is, with 
all the effort and time spent on this technology by 
schools across the United States, has the use of an ASL 
tool enhanced the educational experience in a 
quantifiable or measurable way? There is a lot of 
anecdotal information that points to better feelings by 
students, more interaction, better questions, etc, but little 
has been done looking at quantitative measures. A major 
purpose of this study was to make a preliminary 
determination if there were any. measurable gains noted 
in using an ASL tool to support traditional lecture 
classes. Secondary questions were, at what level does 
the tool use begin to demonstrate some effect on the 
student and course outcomes, and is there a point of 
diminishing returns at whichpoint an increase in the use 
of the tool has no effect on the measured outcomes? 

METHODOLOGY 

This study was based on a convenience sample not a 
randomized experiment. Courses used were those with 
an instructor who was interested in using the ASL 
software, and were not randomly assigned. There was 
no selection of students for the various sections. 
Students who registered for courses were participants 
unless they selected themselves out by dropping the 
course. No course was noted as being a special section 
so there was a random chance for students to be 
involved. All classes described used Blackboard to 
augment the course in some manner, but the level use 
was up to the instructor and depended largely on their 
comfort level with the ASL tool. 



Several measures were used to determine if the use of 
the tool impacted the learning environment. First, what 
was the average grade obtained by students in the 
course? Did it change? To minimize extraneous factors 
only previous sections of the same course taught by the 
same instructor were compared. This information was 
obtained from the final grade report. Second, did the 
average final exam grade change? Even though these 
are all graduate level courses, and exams are typically 
only a small part of the final grade, it was important to 
see if there was a direct link in knowledge as measured 
by tests. Instructors administered the same test to both 
the regular course and the ASL supported course. 
Student exam grades were obtained from instructors. 
Third, did the student evaluation of the course change? 
All students rate courses on the learning experience, how 
well it communicates course information, and how much 
they learned? The same instrument was given to both the 
traditional and the ASL augmented classes. The last 
quantitative measure was how did the students evaluate 
the instructor? Was the instructor seen as being different 
when evaluated by the traditional or the ASL supported 
course? This information was also obtained from the 
course evaluation form.Finally,aqualitative cross check 
was accomplished by looking at student written 
comments discussing the course experience. The 
measurement tools were the usual course evaluation 
form, which was not modified in any way for the study, 
and a special Blackboard survey form thatstudents filled 
out at the end of the term. 

Courses taught during calendar year 1999 were used as 
the sample for the study, and they were chosen for 
several reasons. It was the first year when there were 
enough courses run using Blackboard to obtain a 
reasonably sized sample. The ASL tool had become 
reasonably stable during this period, and both faculty 
and technical staff were fam iliar enough with 
Blackboard that technical problems did not affect the 
classes. Table 1 summarizes the numbers of courses 
taught in the various formats by different departments in 
each term during the year. 

The key to determining the effect of some intervention 
is to have a solid baseline on which to measure 
differences. For this study the base line used was the 
most recent evaluations of the course taught by the same 
instructor, in the same format, without the use of the 
ASL tool. Data was collected on each Blackboard 
course and the corresponding baseline course. 
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TABLE 1 

SUMMARY OF FORMATS 
AND IMPLEMENTATIONS 



Format 


Spring 99 


Summer 99 


Fall 99 


Little Use 


IT (3)* 


None 


IT (2) 
MBA (2) 
MKTG (1) 


Mode rate 
Use 


IT (5) 


IT (8) 


IT (17) 
MBA (3) 
MKTG (2) 


Extensive 

Use 


IT (3) 


IT (3) 


IT (4) 
MBA (1) 



*Numbers in parenthesis are the number of B lackboard 



supported classes run by that department. 

Not all of the Blackboard augmented courses taught in 
calendar year 1999 could be used in the study. Some 
courses were new and had not been taught before, and 
several instructors had not taught some of the courses in 
a traditional lecture class setting. In the extensive use 
courses, three of the eleve n classes co uld not be used for 
these reasons. In the moderate use courses, there were 
seven classes that could not be used. In the low use 
courses, one class could not be used. The result was that 
the course comparison for this study includes seven (7) 
low use courses, twenty-eight(28) moderate use courses, 
and eight (8) extensive use courses. 

RESULTS 

Appendix A presents the raw data for all of the courses 
used in the study. It gives the course grades, course 
evaluations, and instructorevaluations forall forty -three 
(43) courses, and final exam scores for the twenty-three 
courses that had final examinations. The raw numbers 
indicate several points. First, in only a few cases are the 
differences numerically large. Second, in many cases 
the differences do not always show a numeric change 
indicating a positive impact of the technology. This 



means that for some students, the use of the ASL tool 
actually detracted from the learning experience. Lastly, 
the increased use of the tool usually resulted in higher 
numerical evaluations for the course and the instructor. 

Chi-square tests for difference were performed on each 
course. Results were compiled and placed in Table 2. 
Observed Chi-square values and the critical Chi-square 
values for the 0.05 level of significance are shown. None 
of the low usage courses had a significantdifference for 
any of the four quantitative measu res. Of the m oderate 
use courses, the results were variable w ith two measures, 
course and instructor evaluation, showing significant 
increases and two, final exam score, and final grade, 
having no significantdifference. Extensive use courses 
resulted in significant increases in all areas except the 
final exam scores. 

FINDINGS 

Table 3 summarizes the findings of the study. Several 
points can be observed in the information presented. 
First, low levels ofusage of the ASL tool resulted in no 
significanteffect on any ofthe measures employed in the 
study. Even a review of the raw data in Appendix A 
fails to show any numerical indication of a consistent 
effect. Second, for moderate levels of usage, while 
examination and course grades had no significant 
differences, there was a marked variation in the raw 
scores for both these measu res as show n in Appendix A. 
Use of the ASL tool did affect students as both the 
course and the instructor evaluations increased a 
significant amount. Extensive levels of usage had a 
significanteffect on all measures except the final exam 
scores. The scores were all numerically higheras shown 
in Appendix A, but not statistically so. N o reason could 
be postulated for the lack of a significant statistical 
change in that measure when the course grade was 
affected significantly. 



TABLE 2 

CHI-SQUARE TEST RESULTS 





Final 


Exam 


Course 


Grade 


Course 


Eval 


Instruc. 


Eval 




OBS X 


Crit. X 


OBS X 


Crit. X 


OBS X 


CritX 


OBS X 


Crit X 


Low Use 


2.0215 


7.815 


8.512 


12.592 


9.651 


12.592 


6.342 


12.592 


Moderate Use 


8.3865 


26.119 


24.976 


40.113 


40.161 


40.113 


41.241 


40.113 


Extensive Use 


4.4711 


9.488 


14.412 


14.067 


18.104 


14.067 


16.724 


14.067 
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TABLE 3 

RESULTS OF CHI-SQUARE TEST FOR SIGNIFICANT DIFFERENCE 



FORMAT 


Measure 


X-square 


Meaning 


Low Use 


Final Exam 


2.0215 


Not Significant 




Course Grade 


8.512 


Not Significant 




Course Eval 


9.651 


Not Significant 




Instrc Eval 


6.342 


Not Significant 


Moderate Use 


Final Exam 


8.3865 


Not Significant 




Course Grade 


24.976 


Not Significant 




Course Eval 


40.161 


Significant 




Instrc Eval 


41.241 


Significant 


Extensive Use 


Final Exam 


4.4711 


Not Significant 




Course Grade 


14.412 


Significant 




Course Eval 


18.104 


Significant 




Instrc Eval 


16.724 


Significant 



DISCUSSION 

The results do in fact support the hypotheses and 
questions stated earlier. There does seem to be a 
statistically significantdifference in student performance 
as noted by the measures used. This effect is 
ameliorated by the type of interaction that students have 
with the ASL tool thatis used. In low usage course there 
is little student interaction and therefore no impact or 
difference. The numerical differences noted in Appendix 
A could have occurred by chance. 

In the moderate usage courses, while some measures 
show a significant improvem ent, there is high variability 
in numeric scores. Some courses showed high impact, 
somemoderate impact, some low pact, and the rest none. 
This could be because of three factors, which were not 
considered in this study. First, the course subject could 
have an effect. Some subject matter needs extensive 
explanations to facilitate understanding and others do 
not. The second variable is the manner in which the 
course was actually conducted. Although the instructo rs 
were supposed to use the tool in certain ways, those w ith 
different skill sets might not effectively use all of 
Blackboard’s capabilities, which could affect the results. 
Third, student response to the ASL tool was highly 
inconsistent; those with some technical knowledge 
seemed to enjoy it more than students with little or no 
technical knowledge. 

In the extensive usage courses, there was almost 
universal improvement, and all but the final exam scores 
were significant. This level ofusageresulted in the most 



consistent improvement of all of the raw comparison 
data. The electronic based assignments, chats, and 
discussions seem to reinforce the material and create a 
greater impact than merely presenting the data and then 
referring to it. 

Qualitative Comments 

There were no comm ents about the use of B lackboard by 
students who had only low level usage of ASL tool on 
the end of course evaluation. Table 4 shows that the 
time spent with Blackboard by students for the entire 
term was barely more than half an hour per student. The 
evaluation form indicated that most students didn’t use 
it if there was no requirement to use the tool. Students, 
who used Blackboard, read the course syllabus, but only 
a few bothered to review assignments on the web page. 



TABLE 4 

SUMMARY OF BLACKBOARD TIME USE BY 
ASL TOOLUSAGE 



Tool 


Time use 


Low Use 


Mean = 0.5 hr. 


S.D. = .6 hr 


Moderate Use 


Mean = 12.1 hr 


S.D. = 3.4 hr 


High Use 


Mean = 19.6 hr 


S.D.= 2.7 hr 


On-line 


Mean = 26.5 hr 


S.D. = 3.9 hr 



In the moderate level of usage format there was a 
significantly higheruse of the computer-based materials, 
and table 4 shows an increase of over 24 times over the 
low usage courses. The biggest factors for increased 
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usage were the requirement to use the ASL tool for 
several assignments, and the availability of some 
materials for the course only on the Blackboard website. 
Few students noted problems with the course. Student 
comments highlighted the ability to do work for the 
course when they wanted and having more material 
available than from a regular course. 

In the third format, extensive usage of Blackboard, the 
use of lecture materials, notes, and supporting 
documentation on the tool was con siderably higher than 
in the moderate usage format, even though there was no 
significant difference in the amount of supporting 
materials provided. The Blackboard evaluation form 
revealed that as the course went on, students used less 
and less of the supporting materials. E-mail usage 
remained high throughout the term with more 
interactions between students via e-mail than in the 
previous cases. The threaded conversations were used 
heavily initially, decreased in the middle, and increased 
again towards the end of the term. An online chat was 
used weekly, but its usage too mirrored that of the 
threaded discussions. Participation in the threaded 
discussion seemed to be the most effective way to gauge 
and evaluate class participation. 

In the extensive usage classes most students also 
commented that they felt they put in more work than in 
a traditional class, because the instructor was able to ask 
questions on-line and all students had to respond. Itwas 
not possible to skip the readings, and then hope not to 
get called. Several students stated that the threaded 
discussions and the e-mail/chats were the most useful 
part of the course. Students enjoyed links to current 
topics or sources as they reinforced the connection 
between theory and practice and often gave current 
examples of successes and/or failures. They were 
emphatic about the need for more courses using ASL 
tools and the need for full on-line courses. They did not 
indicate any concerns about the limited interface with 
other students or having any problems with access to the 
instructor. Overall students were positive about the use 
of Blackboard as an A SL tool. 

There were somenegative comments too. One complaint 
was the delay with instmctors responding to questions 
and grading assignments. Students seemed to believe 
that if they sent an e-mail out at 2 a.m., it should be 
answered immediately, or by the early morning at the 
latest. Equipment issues were a problem initially. Even 
though everything could be handled through a web 
browser, some students still had equipment problems. A 
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major issue was the use ofaudio and video clips, which 
had been provided by some instructors to add some 
variety to the materials or to make special points for 
lectures. Many students felt they were not worth the 
trouble, and many could not get adequate performance 
from their systems to view some of the video. Students 
who used browsers in the computerlab complained that 
the computers had no speakers, so audio clips could not 
be heard. They felt that many of the lecture notes and 
much of the material presented merely restated material 
from the text. The biggest complaint was that although 
there was a variety of ways to communicate with the 
instructor and students outside of class, most were not 
intuitive and required time and patience to use. Most 
students felt that for a single class that this mightnot be 
a serious problem, but for a continuing program, it 
would be a serious limitation. Even students who made 
these comments were still generally positive about the 
experience. 

CONCLUSIONS 

The information reported here is from a small non- 
randomized sample from one university so no 
statistically valid general inferences can be made. 
Another limitation is that the only instructors included in 
the study are those who vo lunteered to use the AS L tool. 
Several obvious points can be noted. The use of an ASL 
tool is not a panaceaor replacement for a poor instructor. 
When implemented correctly, it can have a significant 
effect in a positivemanner, but if implementedpoorly it 
can have a negative effect. 

Most of the ASL tools currently in use are still not 
mature. Each revision makes products more powerful, 
stable, and functional, but it will be some time before 
they can fully do what the students and instructors need 
and want to fully support on-line courses. Since less 
than 50% of current offerings from ASL tool vendors are 
stand alone on-line courses, it is important to determine 
if the use of the se tools to augm ent traditiona 1 courses is 
educationally effective. The findings reported here 
indicate that the use of Blackboard to augment 
traditional courses can have significant impact on 
student performance and attitude. 

This preliminary study validates the use of four 
quantifiable measures as methods to determine the 
efficacy of the ASL tool use in various formats of 
augmenting traditional classes. Studentuse ofthe course 
material and the time figures in Table 5 clearly show that 
no student spends as much time reading material off the 
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screen as either the instructor or course designer 
believes. Hodes (1993) points out that most students 
prefer the textbook and hard copy notes to reading 
extensive on-line notes and materials. 

Discussions with instructors indicated a high level of 
effort to prepare and teach with the ASL tool at the 
moderate and extensive usage levels. Abernathy (1998) 
and Kerka (1 996) reinforce the point that Skinner (1 968) 
made over 30 years ago that textbooks, lecture outlines, 
film scripts, etc. are of little help in preparing course 
material for asynchronous learning. Instructors also 
noted a requirement to have a good working knowledge 
of the software and limitations. In the first few classes 
software and equipment questions were as common as 
course material questions. These comments compare 
closely with those observed byHarasim (1991) that rapid 
resolution of technical problems by the instructor is 
critical for student learning. 

FUTURE STEPS 

The use of an ASL tool to augment a traditional course 
seems to have an affect on student attitude and 
performance. The wo rk here is a preliminary study, and 
while its findings are interesting and support the stated 
questions, more data must be accumulated and tested 
before any definitive statements can be made. It does at 
least show the potential for the use of ASL tools in this 
mode to improve student performance, and educational 
environment. Several steps are planned afterthis study. 
With the spring semester there should be sufficient data 
to look at individual courses to determine how course 
content impacts the effectiveness of the ASL tool to 
enhance learning. Second, a longitudinal study is 
planned to determine if the course evaluations change 
over time for the same instructors. The possible 
capability of ASLtools to enhance learning in traditional 
courses is an important issue which must be fully 
explored. 
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APPENDIX A 

SUMMARY OF BLACKBOARD USE AND EVALUATIONS 



Tool 


Final 

Exam 


Student Grade 


Course Evaluation 


Instructor Evaluation 


Format 






BlckBrd 


Baseline 


BlckBrd 


Baseline 


BlckBrd 


Baseline 


Low Use 


92. 


92 


3.37 


3.41 


3.12 


3.10 


3.10 


3.08 




91 


91 


3.78 


3.73 


3.45 


3.48 


3.32 


3.29 




94 


93 


3.43 


3.44 


.3.29 


3.28 


3.08 


3.09 




93 


92 


3.45 


3.43 


3.48 


3.46 


3.01 


3.01 




NA 


NA 


3.32 


3.34 


3.32 


3.33 


2.96 


2.95 




NA 


NA 


3.19 


3.24 


3.27 


3.26 


3.21 


3.23 




NA 


NA 


3.35 


3.35 


3.32 


3.34 


3.14 


3.13 

Til 


Moderate 


95 


92 


3:^0 




3.19 


“3t44 


3.41 


J. 1 1 


Use 


93 


92 


3.81 


3.73 ■ 


3.50 


3.79 


3.76 


3.37 




92 


91 


3.78 


3.88 


3.49 


3.54 


3.69 


3.29 




91 


90 


3.97 


3.38 


3.43 


3.36 


3.33 


3.09 




94 


93 


3.29 


3.30 


2.86 


3.24 


3.42 


3.08 




92 


92 


3.90 


4.00 


3.25 


3.47 


3.73 


3.47 




92 


91 


4.00 


3.91 


3.62 


3.60 


2.99 


3.03 




91 


90 


3.86 


3.85 


3.72 


3.68 


3.51 


3.19 




93 


91 


3.94 


3.92 


3.55 


3.56 


3.34 


3.03 




96 


93 


3.88 


3.89 


3.70 


3.69 


3.44 


3.05 




91 


90 


3.76 


3.75 


3.79 


3.75 


3.33 


3.12 




92 , 


90 


3.84 


3.86 


3.35 


3.33 


3.23 


3.03 




93 


91 


3.16 


3.13 


3.27 


3.27 


3.38 


3.17 




94 


91 


3.36 


3.37 


3.34 


3.33 


3.50 


3.11 




NA 


NA 


3.41 


3.40 


3.66 


3.65 


3.66 


3.33 




NA 


NA 


3.47 


3.42 


3.32 


3.31 


3.47 


3.25 




NA 


NA 


3.70 


3.65 


3.21 


3.19 


3.64 


3.24 




NA 


NA 


3.69 


3.67 


3.46 


3.46 


3.58 


3.17 




NA 


NA 


3.29 


3.30 


3.59 


3.58 


3.56 


3.25 




NA 


NA 


3.11 


3.14 


3.33 


3.33 


3.73 


3.40 




NA 


NA 


3.90 


3.86 


3.02 


3.05 


3.27 


3.02 




NA 


NA 


3.94 


3.92 


3.47 


3.45 


3.65 


3.15 




NA 


NA 


3.37 


3.36 


3.54 


3.49 


3.12 


3.01 




NA 


NA 


3.76 


3.75 


3.59 


3.57 


3.53 


3.22 




NA 


NA 


3.24 


3.25 


3.69 


3.68 


3.72 


3.26 




NA 


NA 


3.46 


3.44 


3.23 


3.21 


3.64 


3.41 




NA 


NA 


3.56 


3.51 


3.27 


3.24 


3.52 


3.42 




NA 


NA 


3.61 


3.59 


3.41 


3.40 


3.83 


3.38 


r d. - ’ - 








o ac 






3 7T 


3 19 


c.xi'cnsi VC 


70 


Jj 






J .\JJL 


^ o 






Use 


95 


94 


3.76 


3.86 


3.82 


3.64 


3.83 


3.58 




96 


92 


3.75 


3.76 


3.55 


2.61 


3.55 


3.30 




94 


93 


3.88 . 


3.95 


3.80 


3.46 


3.79 


3.50 




95 


91 


3.84 


3.77 


3.15 


3.51 


3.62 


3.39 




NA 


NA 


3.37 


3.34 


3.71 


3.49 


3.59 


3.48 




NA 


NA 


3.55 


3.42 


3.68 


3.36 


3.74 


3.42 




NA 


NA 


3.90 


3.86 


3.79 


3.47 


3.66 


3.32 
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ABSTRACT 

This paper presents the findings of a study on the use of World Wide Web pages among students at St. Cloud State 
University, Minnesota, USA. The paper explores problems and challenges on campus Web computing and the 
relationships among the extent of Web usage, class level, and overall student academic performance. The paper 
concludes with a discussion on imp roving the effec tivenes s of campus Web services. 



INTRODUCTION 

The rapid advance of Web basedtechnology has created 
a major challenge to Universities and Colleges in 
classroom teaching. It c hanged the nature of information 
technology service used in education (Hong, 1997). At 
the end of the 1990s paperless classrooms began to 
emerge at many colleges and universities (Bialac and 
Glover, 1997). The academic world has seen an 
enthusiastic rush of faculty to the WWW as the new 
mode of interaction with students. Syllabi, projects, 
handouts, and tutorials even entire course and curriculum 
are placed on the Web not only as a new mode of 
distance learningbut also as a supplement to traditional 
mode of face-to-face classroom instruction. According 
to a recent survey of campus officials and IT managers 
across the United States, more thanone-fourth (28%) of 
college courses are supported by a Web page, and thirty- 
ninepercent ofall courses are using Internetresources as 
part of the course requirements (Nelson, 1999). 

The increased use of Web technology as a tool for 
instruction and research by faculty and students creates 
challenges and problems for academic computing 
management. Schools are facing challenges of 



upgrading and expanding computing facilities. Faculty 
support and instructional integration have become the 
most important IT fiinctions in recent years (Thiele, 
1999). The problems that affect student effective 
learning include lack of knowledge on how to use 
computers, unstable technology in use, slow Web access 
speed, lack of available hardware and software among 
others. Ultimately, to improve effectiveness and 
maximize the benefits of Internet usage in education, 
solutions are needed to minimize these problems and to 
increase the satisfaction of faculty and students who use 
the Web technology. 

The purposes of this study are (1) to identify campus 
Web related problems from student perspective (2) to 
conduct a preliminary investigation into the relationships 
among student computer skills, frequency of reported 
Web related proble ms, stu dent pe rform ance (G PA). It is 
hoped that the study results will help us understand the 
types of Web related problems that college students 
experience, the factors might contribute to the 
difficulties, and whether frequency of Web usage is 
related to student overall performance. The paper is 
organized as follows. After a review of current issues in 
university Internet services, a brief descriptions of the 
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computing facilitiesof the two campuses in the study are 
provided. Then research methodology and findings are 
discussed. The paper concludes with a summary and 
some suggestions for improving campus Internet 
services. 

CURRENT ISSUES IN 
UNIVERSITY INTERNET USAGE 

Technology resources like e-mail, FTP, the World Wide 
Web (WWW ), and multimedia are increasingly common 
components of the instmctional experience for college 
students. Additionally, growing numbers of campuses 
now have a computer competency or computer 
instruction requirement for all their undergraduate 
students. According to the Kenneth C. Green (1997), 
visiting scholar at the Center for Educational Studies at 
Claremont Graduate University, "Technology resources 
are becoming an increasingly important component of 
the instructional experience, across all fields and all 
types of institutions." The use of information 
technology (IT)as an instructional resource is highest in 
research universities. However, Green notes that the 
gains have occurred in all types of colleges and 
universities: "Although students in universities are more 
likely to encounter IT resources in th eir classes than their 
peers at four-year colleges or community colleges, the 
survey data document the expanding use of IT resources 
across all sectors of American higher education." 

Seemingly endless streams of Internet users present 
problems of traffic control and overall accessibility. 
Already, the Internet is frequently overloaded with 
electronic messages. Demand for ever larger e-mail 
volume, multimedia content, real time broadcasting, 
electronic meetings, download of free software, music, 
and games, and a host of developing technologies and 
applications will continue to tax the existing 
infrastructure of US colleges and universities. 
Brownouts and overloaded networks are increasingly 
common as traffic on the Internet continues to grow 
faster than bandwidth capabilities. For colleges with 
limited budgets and computing resources, keeping up 
with the increasing demand by bandwidth hungry 
students for Internet access is particularly difficult. 
Computing resources at college and universities often 
lag behind the increase in web access from students. 

This situationhas not gone unnoticed. The recent survey 
of chief academic computing officers respondents by 
The Campus Computing Project (1999) also identified 
"assisting faculty integrate IT into instruction" and 



"providing adequate user support" as the top IT 
challengesconfrontingtheir institutions. Almost40% of 
institutions identified "instructional integration" as the 
single most significant IT issue in their institutions, 
followed by 28% citing "providing adequate user 
support." IT financial planning for replacing aging 
hardware and software ranked third at 14%. 

INTERNET ENVIRONMENT AT 
ST. CLOUD STATE UNIVERSITY 

St. Cloud State University (SCSU) is a comprehensive 
university located about 65 miles northwest of 
Minneapolis/St. Paul Minnesota. The University 
consists of seven colleges and a graduate school and 
offers over 60 baccalaureate degrees, several masters 
degrees, and one doctoral program . St. Cloud State 
University currently serves a student population of 
14,000 and has approximately 1000 employees. Class 
rooms, computing laboratories, departmental and faculty 
offices, and dormitories are all wired into the fiber optic 
based campus network. As with most colleges and 
universities demand for Internet services at St. Cloud 
State University has increased dramatically within the 
past few years. 

St. Cloud State University is served by MNET, which 
provides Internet services to Minnesota government 
agencies and educational institutions. A T1 
communicationsline connects St. Cloud State University 
to the regional Internet hub in St. Paul. Hundreds of 
computers on campus are cabled directly into the campus 
network via Ethernet adapters. Token Ring LANs 
connect buildings and laboratories on campus. A bank 
of 330 highspeedmodems provides dial-up access to the 
Unive rsity network. 

While there have been numerous discussions over the 
years, there are currently no university-wide standards 
for desktop hardware, operating systems, applications 
software, Web browsers, or email packages. However, 
good support is available for the most popular 
configurations and applications those who choose to 
deviate from the “beaten path generally find themselves 
without support. Most of the open computing 
laboratories are maintained by Academic Computing 
Services, the others are maintained by department or 
college. 

Centrally, the University maintains two general usage 
multiprocessor Digital Equipment Corporation Alpha 
class (one runs the VM S operating system; the otherruns 
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UNIX) email and Web servers. Both machines are 
available for faculty and students e-mail and Internet 
usage. Basic user training in e-mail operations, web 
surfing and home page development is provided at no 
cost to students and faculty by Academic Computing 
Services; advanced training is provided through 
numerous courses offered by v arious departm ents aero ss 
campus, primarily to their own major and minor 
students. For maximum flexibility and control, many 
departments now maintain their own (primarily 
WindowsNT 4.0) servers, which are also connected to 
the campus topology. A few individual faculty members 
have installed server software on their own computers 
allowing students to access files and Web p ages directly 
from machines in the faculty member’s office. 

Many of the needed Web development tools are readily 
available to faculty and students: a variety of plain text 
and HTML editors. Front Page, Java Script authoring 
tools, and Java compilers. An open multimedia lab 
offers digital capture, production, and editing 
capabilities. 

METHODOLOGY 

Study Design 

The study is designed to proceed in three steps: (1) 
identify major problems, (2) analyze thefrequency ofthe 
problems, (3) test hypotheses. 

Null Hypotheses 

Based on common pedagogical knowledge that web 
pages can be effectively used in teaching at colleges and 
universities, the following null hypotheses (Ho) are 
formulated: 

1. Class level does not influence the frequency of 
weekly web accesses. We commonly believe that the 
higher the class level, the higher the needs to access 
the web for class projects and research. 

2. Frequent visit to the web does not have impact on 
GPA and vice versa. Frequent Web usage may 
either help students’ performance or decrease 
students' performance, depending on what the 
students use Web for. Spending time on Web for 
playing games and chatting will negatively affect 
students' performance. On the other hand, students 
who use Web for researching term papers and 



looking for referencesand online academic help are 
expected to perform better. 

3. Slow web access does not impact the frequency of 
web usage. Does perception of slow access prevent 
students from going into the web frequently? 

4. Problems with hardware and software do not impact 
the frequency of web access. Do problems 
associated with hardware and Internet software 
prevent students from accessing the web frequently? 

5. Recognition of Web’s value in research does not 
lead to frequent web access. Do students who 
appreciate Web’s value in research access the web 
more frequently? 

6. Rating of University’s Internet Service is not 
influenced by student’s class level. Do students rate 
Internet service provided by the university 
influenced by student’s class level? 

7. Appreciation of Web value in research is not 
influenced by student’s class level. Do students at 
higher class level appreciate value of the Web more 
than students at lower class level? 

8. Rating ofUniversity’sInternetand Web Services is 
not influenced by student’s GPA. Do students who 
have high GPAs appreciate University’s service 
more than students who have lower GPA? 

9. Recognition of Web’s value in research is not 
influencedby student’s GPA. Do students who have 
higher GPAs appreciate Web values’ more than 
students who have low GPAs? 

Survey Questionnaire 

Two brainstorming sessions with students were 
conducted to produce a list of possible problems that 
students encountered while using the World Wide Web 
service provided by the universities on and off campus. 
The final list of these problems was used to develop the 
questionnaire. Items in the survey fell into three areas: 
(1) student academic background and frequency of web 
access, (2) problems thatstudents faced, and (3) student 
level of satisfaction. Respondents were asked about the 
frequency of occurrence of the problems they 
encountered by selectingone ofthe five choices: never, 
rarely, sometimes, usually, and always. They were also 
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asked to rate their satisfaction level ranging from worst 
(lowest) to excellent (highest) . 

Conducting the Study 

During the period from the Fall semester 1997 to Spring 
semester 2000, 317 students of several classes at St, 
Cloud State Univers ity were surveyed. Participants w ere 
told in advance that the anonymous survey responses 
would be confidential. With some motivation to getout 
of the class early after completion of the survey, all 
students participated. Data collected were tabulated and 
analyzed using Statistical Analysis System (SAS) 
available on the University's VAX computer. 

FINDINGS 

Analysis of the results showed that the great majority of 
the respondents (7 1 ,7%) are seniors, followed by Juniors 
(2 1,3%) and graduated students (6%), Nearly sixty nine 
percent (69 %) of the respondents have an average GPA 
above 3.0, The average number of web accesses per 
week is 1 9.48 times. 

Overall, the results showed satisfaction in using Internet 
facilities (Table 1). Sixty percent (60%) of the 
respondents rated university web services good to 
excellent and 51 percent gave good to excellent ratings 
for their research work. 



TABLE 1 

INTERNET USAGE RESPONSES 



Usage 


Number of 
Responses 


Percent 


Web Services 
Poor or below average 


10 


3,7 


Average 


97 


35,9 


Good 


134 


49.6 


Excellent 


29 


10,7 


Values in doing research 

Poor or below average 


24 


9.0 


Average 


115 


39.5 


Good 


114 


42.9 


Excellent 


23 


8.6 



TABLE 2 

FREQUENCY OF REPORTED PROBLEMS 



Problems 


Frequency 


Percentage 


Slow Access Speed 




Rarely or Never 


16,2 




Sometimes 


40,6 




Always or Usually 


43,2 


Lack of Color Scanner 




Rarely or Never 


35,2 




Sometimes 


29,2 




Always or Usually 


35,6 


Lack of Image Editor 




Rarely or Never 


30,7 




Sometimes 


32,0 




Always or Usually 


37.3 


Poor Text Editor 




Rarely or Never 


37.4 




Sometimes 


32.3 




Always or Usually 


29.3 


Web Development Language Problems 




Rarely or Never 


21.9 




Sometimes 


40.1 




Always or Usually 


38.1 


Outdated Support Software 




Rarely or Never 


29.3 




Sometimes 


41.7 




Always or Usually 


29.0 


Problem in Upgrading Web Browsers 




Rarely or Never 


30.8 




Sometimes 


30.1 




Always or Usually 


39,1 


Free Speech Limitation 




Rarely or Never 


40.6 




Sometimes 


28.2 




Always or Usually 


31.2 


Problems with FTP 




Rarely or Never 


31.4 




Sometimes 


38.3 




Always or Usually 


30.4 


Slow Network 




Rarely or Never 


18.1 




Sometimes 


37.5 




Always or Usually 


44.4 
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Table 2 
(continued) 



Problems 


Freauencv 


Percentafie 


Slow CPU 




Rarelv or Never 


22.8 




Sometimes 


32.7 




Alwavs or Usuallv 


44,5 


Slow Modem 




Rarelv or Never 


23.4 




Sometimes 


30.8 




Alwavs or Usuallv 


45.8 


Problems with Searc 


1 Eneines 




Rarelv or Never 


41.9 




Sometimes 


30.6 




Alwavs or Usuallv 


27.4 


Outdated Internet Software 




Rarelv or Never 


31.7 




Sometimes 


41.8 




Alwavs or Usuallv 


24.5 


Not Enoueh Comouters in the Lab 




Rarelv or Never 


37.8 




Sometimes 


30,4 




Alwavs or Usuallv 


31,7 


Crashes 




Rarelv or Never 


32,5 




Sometimes 


38,5 




Alwavs or Usuallv 


29iL 



Despite the generally favorable results, problems in 
accessingweb pages are evident as presented in Table 2, 
Chief among these are the compla ints aboutthe slowness 
which include high occurrence (always and usually) of 
problems caused by modem (45,8% of respondents), 
processor (44.5%), network (44.4%), and total access 
(43,2%), Other problems cited at always or usually 
occurrence levels include upgrading web browser to 
current level (39,1%), language for web page 
development (38,1%), lack of an image editor such as 
Photoshop (37.3%), not enough computers available in 
the labs (3 1.7%), unable to exercise free speech (31,2%), 
problems with file transfer facility (30,4%), crashes 
(29%), outdated support software such as MS Office or 
Image Editor (29%), poor search engines (27,4%), and 
outdated Internet software such as Telnet, FTP, etc, 
(24,5%). 



Correlation Analyses 

Based on Spearman rank correlationtests of hypotheses, 
Hq! r = 0 (i,e, there is no correlation between pairs of 
class levels, frequency of weekly accesses, software 
problems, value of research, etc.) with p < ,05, this study 
accepts the some of the above Null hypotheses and finds 
no correlation between: 

1 . GPA of students and frequency of web access. There 
is no significantcorrelationbetween web access and 
academic success ofstudents measuredby GPA, 

2. GPA and the value of the University’s Internet 
Service, There is no difference in the evaluation of 
University’s Internet service between students 
having high GPAs and those having low GPAs, 

3. GPA and the recognition of the value of Web in 
research. Students appreciate Web value in research 
regardless of their level of academic performance, 

4. Class level and the appreciation of University’s 
Interne t and W eb Serv ices, 

5. Class level and the appreciation of Web facility in 
researc h. 

However, at 5% significant level, some of the above 
Null hypotheses are rejected: 

1, Students of higher classes reported less problems 
with slow network than students of lower class 
levels network (corr= -,2164, p=,0001, n=3 13), 

2, Students of higher class levels made more access to 
the network than students of lower levels (corr= 
0,14522, p=0,118, n=299), 

3, Students who reported less problems with hardware 
and software access the system more frequently. The 
rank correlation results between frequency of access 
and: (1 ) slow access speed (corr=-0, 14604, p=,0 125, 
n=292), (2) File transfer facility (corr= -.13386, 
p=,0233, n=287), (3) slow network (corr=-, 21016, 
p=,0002, n=300), and (4) slow CPU (corr= -,14003, 
p=,0165, n=293), 

4, Students who made frequent access also complained 
more problems with free speech caused by 
limitations imposed by the University and the 
government (corr=, 19937, p=,0008, n=282). 
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5, Students who made frequent web access reported 
more problems with the shortage of image scanner 
(corr=.15312, p=.0l00, n=282). 

6, Students who appreciate the value of web sites in 
research work made more frequent access than the 
students who did not (corr=, 1 5980, p=.0112, 
n=251). 

7, Students with higher overall GPAs reported more 
frequentproblems with outdated and old versions of 
software (corr=, 1 1842, p=.0401 , n=30 1). 

9, Students with higher overall GPAs also complained 
more problems with free speech caused by 
limitations imposed the University and the 
government (corr=. 12049, p=. 0393, n=293) 

10. Students with higher overall GPAs also reported 
more problems with search engines (corr= .12297, 
p=.03 21, n= 3 04). 

IMPROVING UNIVERSITY’S 
INTERNET SERVICES 

Better Internet hardware, current software, and services 
are needed to improve the productivity, efficiency, 
success and satisfaction for student users at University, 
While class level does not impact the use and 
appreciation of web technology, frequent problems with 
web facility and supports seem to discourage students 
from accessing the web. For the problems with theuse of 
file transfer facility, it is suggested that students should 
be taught how to use it. The use of file transfer utility 
requires students to know basic commands, file 
directory, and options (binary, ASCII), Wrong target 
directory and options could spell troubles to new 
students. Furthermore, students of higher GPAs seem to 
exceed the current capability of software and are likely 
to find Internet and Web software in the lab outdated. 
Understanding of free speech limitation imposed by the 
University and government could be improved by a 
course in Computer Ethics and Laws, While principles 
for effective use of application software in the work 
place and classrooms have been well documented, 
certain basic, but often overlooked, elements can and 
should be introduced into Internet services at University. 
These include improving access speed, faster network, 
better user training, better tools, electronic commerce 
capacity, better network and data security, and frequent 
upgrade of software. Unfortunately, many of the 
problems such as limitation of search enginecapabilities. 



Web development languages, remote server overload, 
bugs that caused browser crashes, etc, are beyond the 
control of University’s computing services department. 

CONCLUSIONS 

This study has confirmed some existing belief, 
suspicions, and added to some existing knowledge in 
Web usage in classroom. Better hardware, up-to-date 
software, training, and services are needed to improve 
the efficiency, success, learning, and satisfaction for 
students' at St, Cloud State University, Frequent 
problems with web facility and supports also seem to 
discourage students from taking advantage of the web 
capabilities in learning. It is suggested that students 
should be well trained in Internet and Web facilities 
before using the University’s computer labs for class 
assignments. Clearly, with the rapid advance in Internet 
and web technologies, opportunitiesexist in curriculum 
development to facilitate and enhance the learning and 
effectiveness of University’s computer services. 
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APPENDIX 



WWW Survey 

All responses are anonymous and confidentia 1. 

/. Student's background. Please circle appropriate number or letter. 

1. Your class standing: 5: Graduate, 4: Senior, 3: Junior, 2: Sophomore 1: Freshman 

2. Number of times you access the Web pages per week 

3. Your major GPA is about: A:3. 5 ‘4.0, B: 3.0-3.49, C:2.5-2.99, D: 2.0-2.49, E: Below 2.0 

4. Your overall GPA is about: A:3.5 -4.0, B: 3.0-3.49, C:2.5-2.99, D: 2.0-2.49, E: Below 2.0 

II. Frequencies of problems encountered in your home page development: Please circle A: Always, B: Usually C: 
Sometimes, D: Rarely, E: never 

5. Computer and/or network speed is slow A B C D E 

6. Lack of color scanners available caused major problems A B C D E 

7. Lack of good software to edit image and convert image type (ie. from JPG type file to GIF or BMP type file) A 
B C D E 

8. Text editor on host computer (Unix) is difficult to use A B C D E 

9. Text color is not flexible to be changed A B C D E 

10. Language that supports interactive page such as Java or CGI is limited and difficult to use A B C D E 

1 1 . Network software is back-versioned and outdated A B C D E 

1 2. Download new version of network access such as Netscape is difficult to connect and takes too much time A B 

C D E 

13. Uploading/downloading text and software from PC to host computer is difficult A B C D E 

14. Cannot develop or link web page that is provocative or politically incorrect (ie. problem with free speech) which 

cause problems in development of quality web page A B C D E 

III. Frequencies of problems encountered when accessing other web pages: Please circle A: Always, B.Usually C: 
Sometimes, D.Rarely, E: never 

15. Slow network connecting/transferring data from and to other hosts A B C D E 

16. Computer CPU/video used in development is slow (486 CPU 33 MHz or less) A B C D E 

17. Connecting modem is slow (14,400 BPS or less) A B C D E 

18. Difficult to locate/find target pages (using Yahoo, Webcrawler, etc) A B C D E 

19. Software is too outdated (old version) to take advantage of latest web page technology A B C D E 

20. Not able to find computer that can access web pages in the computer lab when needed A B C D E 

21. Web page crashes during connection/access A B C D E 

22. Your evaluation of WWW services available at this university: 

A: Excellent, B: Good, C: Average, D: Below Average, C: Poor 

23. Your evaluation of WWW services to help you in research paper: 

A: Excellent, B: Good, C: Average, D: Below Average, C: Poor 

24. Please write below other problems that you have in using WWW pages: 



25. Please write below other problems that you experienced in the development of your own home page: 



126 



Proceedings of the 15^ Annual Conference of the International Academy for Information Management 




135 



TEACHING SYSTEMS ANALYSIS AND 
DESIGN TO A MIXED CLASS OF 
CS AND BIS MAJORS 
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Extended Abstract 



This paper presents the observations of the author in 
teaching Systems Analysis and Design to a combined 
class, comprising of Computer Science and Information 
Systems third year students, The findings are based on 
an interpretivist approach, The students from the two 
streams have different backgrounds, differentattitudes to 
the course, and different involvement, which lead to 
different success rate in the course. 

Systems Analysis and Design is one of the most 
important courses in the IS curriculum, Atthesametime 
there is a discussion to what extent this course should be 
included in the CS curriculum if at all. It must be noted 
however that the emphasis of the CS degree at the 
University of Natal, Pietermritzburg is more on the 
practical concepts, bringing it closer to some Software 
Engineering programmes, 

CLASS BACKGROUND 

The class consists of 53 students, 25 majoring in 
Computer Science, and 28 majoring in Business 
Information systems. The major in Computer Science is 
offered by the Faculty of Science, All CS students have 
a strong background in Mathematics, Statistics and 
Programming, The major in Business Information 
Systems is offered in the School of Business and all 
students have a good background in Economics, 
Management, Accounting, and good understanding and 
exposure to the business practices. They have relatively 
good exposure to programming. The course includes 26 
lectures, seven tutorials, and four three-hour practicals in 
the computer lab. 



The two groups have different attitudes towards the 
course, according to the questionnaire filled by the 
students at the beginning of the semester. The CS 
students consider the course as an intrusion to the CS 
curriculum. This is a worrying indicator showing a 
culture being promoted by the rest of themodules in CS, 
The BIS group has high expectations about the subject, 
which is seen as the one that will bring them closer to 
their future working practices. 

BRIEF DESCRIPTION OF 
SOME ASPECTS OF THE COURSE 

Lectures 

One of the intentions of the lecturer in this course was to 
create an interactive discussion atmosphere during 
lecture time. 

Case Studies 

The use of case studies was one of the tactics often used 
in this course. Many CS students felt uncomfortable to 
be left without clear one dimensional model answers, 
being used to single solution answers within their 
science background. Multiple views, as an opinion 
concept, was not known to them. 

Literature Survey 

The first of the two projects was a literature survey on 
differenttopics related to Systems Analysis and Design, 
The groups had a choice of topics as Information 
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Engineering, Business Process Reengineering, Joint 
Application Development, Prototyping etc. During the 
three years of study this was the firsttextual assignment 
for the CS students. It was also the first oral presentation 
for most of the Science students and it was met with a lot 
of anxiety. 

Analysis and Design of a Business Information 
System 

This was the second assignment which revealed further 
differences between the two streams but to a lesser 
extent than at the beginning of the course. 



The paper expands on the above issues. It relates the 
tasks of the course to the typical outcomes within the 
most recent approved IS curricula by ACM, DPM A and 
other bodies. It com pares the co ntent of the c ourse with 
therequirements ofpublished Software Engineering and 
Computer Science curricula. In conclusion the paper 
identifies directions for further work to improve this 
course towards being a foundation for creating systems 
developers able to design complex information systems 
no matter what their original background and limitations 
could be. 
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ABSTRACT 

Information technology (IT) educators face the two converging trends of globalization and IT human resource 
scarcity. These two trends challenge IT educators to understand how skill requirements differ in the global economy. 
In an attempt to facilitate this understanding, the study reported here investigates the question of how the skill 
requirem ents for recent IT graduates vary across two countries; Australia and the United States, Employers in each 
country were surveyed to determine the importance they place on various skills when evaluating entry-level IT 
employees. Results indicate that there area number of similarities across th e two cou ntries. However, further analysis 
reveals that while the relative importance of various categories of skills are similar across the countries, the 
magnitude of differences in the mean ratings of various skill categories differs. The d ifferences in mean ratings tend 
to be more significant in Australia than in the United States , The results of this study hold implications for both 
employers and IT educators. 
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INTRODUCTION 



As information technology (IT) progresses into the 21®‘ 
century, two trends that are converging hold particular 
importance forIT educators. First, information systems 
are becoming increasingly global in nature. In fact, it has 
been stated that global information systems are now 
more the norm than the exception (Collins, 1996). 
Second, satisfying the demand for properly skilled IT 
professionals is increasingly difficult. When combined, 
these trends represent an important challenge for IT 
educators — how to provide IT program graduates who 
posses the skills necessary to succeed in today’s global 
environment. 

In order to take a step toward he Iping educators meet this 
challenge, this paper abstracts an ongoing research 
project that compares the importance employers in 
Australia and the United States place on various IT- 
relatedskills whenevaluating entry-level IT employees. 
The results of the completed research may 1) help 
educators betterunderstandIT employers’ requirements 
for entry-level IT human resources, 2) help IT 
practitioners gain insight into differencesin employment 
environments that may affect their global IT teams, and 
3) provide IT students who wish to pursue global 
opportunities with information on how skill 
requirements may vary between countries. 

PURPOSE OF THE STUDY 

The purpose of the research -in-progress described here 
is to identify and understand differences and similarities 
in the importance of various skills as viewed by 
employers of entry-level IT professionals in Australia 
and the United States. While the literature includes a 
number of studies that focus on IT-related skills (e.g. 
Cheney, et al., 1990; Leitheiser, 1992; Lee et al., 1995; 
Nord & Nord, 1995; Athey & Wickham, 1995-1996; 
Van Slyke et al., 1998), there has been little attention 
paid to two particular aspects of IT-related skills, 1) the 
importance of skills for entry-level positions and 2) 
comparisons of skill importance across countries. The 
research described here addresses both ofthese neglected 
areas by comparing the perspectives of Australian and 
North American employers’ perceptions of the 
importance of various IT-related skills. 

METHODOLOGY 

A survey was administered to employers of recent 
graduates of university IT program in two countries, 
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Australia and the United States. Subjects consist of 
individuals involved in the recruiting of entry-level IT 
professionals. Note that th e respondents are diverse with 
respect to the size of the organ ization they represent, the 
geographic location of the organization, and the industry 
in which the organization operates. While this does not 
guarantee a truly random sample of all IT employers, it 
should provide fora sample that is representative of the 
population of IT employers in Australia and the United 
States. To date 25 Australian and 33 United States 
responses have been received. 

The development ofthe survey instrument is reported in 
Van Slyke, Kittner and Cheney (1998). Briefly, the 
survey presents a number of skills and asks subjects to 
rate how important each skill is when evaluating 
applicants for entry-level IT positions w here a Bachelors 
degree is required. The rating is done on a scale of 1 (not 
important) to 5 (critical). In addition, subjects are 
encouraged to add skills to the list. Skills were 
categorized by a number of IT professionals, and these 
categories were used to group the skills on the 
questionnaire. The categories areshown in Table 1 along 
with a number of examples of skills from each category. 
Note that the example skills are non-exhaustive. They 
are simply intended to provide the reader with sense of 
the skill category. 

RESULTS 

Analysis of the surveys received to date indicate that 
there is general agreement in the ratings given to the 
various skill categories. With one exception, there were 
no statistically significant differences in the category 
mean ratings between the Australia and United States 
respondents. Table 1 shows the mean rankings and 
standard deviations for each skill category by country. In 
addition, significance levels (p) are given. 

As can be see n from T able 2, employers in Australia and 
the United States seem to view the importance of skills 
similarly. With only one exception (Non-technical 
skills), there are no statistically significant differences 
between the category mean ratings of employers in 
Australia and those in the United States. 

Interestingly, the only skill category to show a 
significant difference in the mean ratings was Non- 
technical skills. There is general agreement in the 
literature that non-technical skills are very important to 
IT professionals. Further, in both countries, employers 
place more importance on non-technical skills than they 
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do on any category of technical skills, which confirms 
the results of the prior studies. It is also interesting to 
note that there was general agreement among the 
respondents within each country as to the importance of 
non-technical skills. The standard deviations for both 
countries are relatively low. In fact, the Non-technical 
category had the smallest standard deviation for the 
United States respondents. The results related to the 
Non-technical skills category may be interpretedtomean 
that although employers in both countries feel that non- 
technical skills are very important in evaluating new IT 
graduates, the Australian employers find these skills to 
be less important than do employers in the United States. 


In attempting to understand the results it is interesting to 
look at the absolute differences in ratings for the two sets 
of employers. In most cases, not only were there no 
differences of statistical significance, there were only 
very small differences in terms of absolute ratings, 
which are shown in Table 3. The mean ratings of only 
two categories. Operating Systems, and Non-technical 
skills, differed by more than 0.50 (on a five-point sc ale). 
This tends to indicate that, in general, employers in the 
two countries feel similarly about the importance of 
different categories of skills. 



TABLE 1 

SKILL CATEGORIES AND EXAMPLES 



Category 


Exam pie skills 


General information systems 


Systems analysis and design 
Databa se conce pts 
Prototyping 
Telecommunications 


Programming languages 


COBOL 

C++ 

Visual Basic 
Java 


Operating systems 


Unix 

Windows 95/98/2000 

JCL 

VMS 


Networking 


NT Server 
Novell Netware 
TCP/IP 

System security 


Database management systems 


DB2 

Oracle 

Microsoft Access 
SQL 


Other information systems skills 


EDI 

PC applications 
Lotus Notes 
AI development 


Business skills 


Understand business functional areas 
User interviewing 
Feasibility analysis 
Project management 


General skills 


Analytical ability 
Verbal communication 
Written communication 
Teamwork 
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TABLE 2 

RESULTS BY SKILL CATEGORY 



Category 


Australia 

Mean 


Australia 
Std. Dev. 


U.S. 

Mean 


U.S. 
Std. Dev. 


Significance 


General IS 


3.42 


0.57 


3.58 


1.18 


0,533 


Programming Languages 


2.76 


0.95 


2.8^ 


1,44 


0,763 


Operating Systems 


2.54 


1.23 


3.09 


1,50 


0,138 


Networking 


3.46 


0.42 


3.70 


1,10 


0,702 


Non-technical 


4.16 


0.45 


4.73 


0,71 


<0,001 


Business-Related 


3.17 


0.29 


3.40 


1,17 


0,341 



TABLE 3 

ABSOLUTE DIFFERENCES IN MEAN RATINGS 



Category 


Australia Mean 


U.S. Mean 


Difference 


General IS 


3.42 


3.58 


-0,160 


Programming Languages 


2.76 


2,86 


-0,100 


Operating Systems 


2.54 


3,09 


-0,550 


Networking 


3.46 


3.70 


-0,240 


Non-technical 


4.16 


4.73 


-0,570 


Business-Related 


3.17 


3.4C 


-0.23C 



Results for the two countries are also similar in the 
relative importance of each skill cate gory. Table 4 shows 
the ranking of each skill category from most important 
to least importantas measured by the category mean. For 
convenience, the mean rating is given in parentheses. 

Once again, the results are similar for both countries. 
The top four rankings are the same for Australia and the 
United States. However, the rakings in the two last spots 
are reversed, with Australia showing Programming 
Languages in the fifth position and Operating Systems 
sixth, and the opposite for the United States. 

In order to gain insight into the relative importance of 
the different skill groups, a series of t tests were 
performedthattestedfor significant differences between 
the mean ratings of pairs of skill categories for each 
country. The categories were paired based on therelative 
rankings shown in Table 4. For example, the Australian 
mean rating of skills in the Non-technical category was 
tested against the mean rating for the Networking 
category. Table 5 provides the results of these tests. 



Examination of Table 5 provides some interesting 
contrasts between the results from the two countries. In 
both countries, there is a highly significant difference 
between the Non-technical and Networking categories. 
The results for each country are also similar for 
Networking vs. General IS, which is non-significant. 
However, differences in the results for the two countries 
emerge further down the list. For the Australia group, 
there is a borderlinesignificantdifference in the General 
IS and Business-Related ratings, while the difference 
between the categories is highly non-significant for the 
United States group. In order to be more complete in the 
analysis, an additional t-test was performed to test 
whether the United States Business-Related and 
Operating Systems mean ratings were different. The test 
reveals borderline significance (p = 0.082). This is 
somewhat similar to the Australian result, which 
indicatesthatthere is a significantdifference in the mean 
ratings of these two categories (p = 0.045). Neither 
country displayed a significant difference in the mean 
ratings of the Programming Languages and Operating 
Systems categories. 
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TABLE 4 

SKILL CATEGORY RANKINGS 



Rank 


Australia 


United States 


1 


Non-technical (4.16) 


Non-technical (4.73) 


2 


Networking (3.46) 


Networking (3.70) 


3 


General IS (3.42) 


General IS (3.58) 


4 


Business-Related (3.17) 


Business-Related (3.40) 


5 


Programming Languages (2.76) 


Operating Systems (3.09) 


6 


Operating Systems (2.54) 


Programming Languages (2.86) 



TABLE 5 

INTER-CATEGORY TESTS 



Australia 


Significance 


United States 


Significance 


Non-Technical vs. Networking 


< 0.001 


Non-Technical vs. Networking 


<0.001 


Networking vs. General IS 


0.826 


Networking vs. General IS 


0.671 


General IS vs. Business-Related 


0.057 


General IS vs. Business-related 


0.536 


Business-Related vs. 
Programming Languages 


0.045 


Business-related vs. Operating 
Systems 


0.353 


Programming Languages vs. 
Operating Systems 


0.483 


Programming Languages vs. 
Operating Systems 


0.527 



CONCLUSIONS 

Before discussing any conclusions that can be drawn 
from the results of this study, it is importantto note that 
the results reported here are preliminary and are based 
on relatively small samples. Care should be taken when 
basing actions on these results. Additional research is 
necessary in order draw strong conclusions. 

It is clear that according to the results of this study 
employers in both countries seek IT graduates with solid 
non-technical skills. In both Australia and the United 
States, the Non-technicalcategory had the highest mean 
ratings. Further, in both countries, the differences in the 
mean ratings of Non-technical skills and those of the 
next highest-rated category (Networking) are highly 
significant. The message from employers is that non- 
technical skills are critical, regardless of the country. 
This result lends credence to earlier, single-country 
studies that found “soft” skills to be very important to 
success in the IT world. Our findings indicate that the 
importance of these skills is not limited to a single 
country. Further, these skills not only are important to 
the success of IT profession als, they are critical to entry 
into the field. 



Regardless of the perspective, significant differences, 
absolute differences, or relative rankings, it seems 
reasonable to conclude that there are a number of 
similarities in what employers in both countries desire 
from entry-level IT professionals. In both Australia and 
the United States, non-technical skills are highly sought 
after, as are networking and general informationsy stems 
skills. Programming skills andoperatingsystems-related 
skills are much less important when evaluating entry- 
level IT professionals in both countries. 

It is important to recall, however, that despite the 
similarities in the results from the two countries, it would 
be incorrect to state that the employers in Australia and 
the United States view the importance of various skills 
exactly the same way. The analysis of inter-category 
differences (see Table 5) reveals that there are more 
pronounced differences in the mean category ratings in 
Australia than in the United States. 

As information systemsbecomemore global in scope, IT 
project managers should take heart from the results of 
this study. Employers in two distant countries view the 
importance of skills similarly. This bodes well for 
project managers who must deal with IT professionals of 
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different nationalities. If the perceptions of those who 
employ IT graduates influence the educational programs 
that create the graduates, there will be similarities in the 
skill sets of graduates from Australia and the United 
States. The similar skills sets should make managing 
human resources in global IT projects easier. 

In addition, there are implications for educators. It seems 
thatthe desiresof employers are somewhat homogenous 
across countries. This may mean that educators in 
different countries who wish to have their students 
interactshould be able to develop programs that meet the 
needs of employers in multiple countries. The 
similarities may also m ake it easier to design educational 
programs that allow students to spend time in other 
countries, without fear of failing to meet the needs of 
employers in the home country. For example, Australian 
students who spend time studying in the United States 
would find that the importance of the skills gained in the 
experience would be viewed similarly by Australian 
employees. Of course, this assumes that there is 
correspondence between employers’ desires and 
curricula. 

Paintinga comprehensive picture of how the importance 
of skills varies across countries requires considerable 
further effort. A wider sample of employers in both 
subject countries needs to be surveyed. In addition, 
additional countries should be studied to find whether 
the results found in this study hold. Finally, surveys are 
limited in the depth of understanding they can provide. 
Interviews with employers (which are currently being 
conducted as part of this study) may provide deeper 
insights into the meaning of the results reported here. 
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ABSTRACT 

Student MIS organizations can be a valuable asset in achieving the goals of a university's MIS program. In this 
preliminary investigation, we surveyed student MIS organizations using the Doty, etal, measures of organizational 
effectiveness. Thirteen valid surveys were returned and the web sites of those student organizations were content 
analyzed. The analysis revealed that successful student organization websites implemented these structural 
components: mission sta tement, on line link to U niversity Career Center, online event calendar, online list of officers 
and committees, online membership application, and online social event announcements. 



INTRODUCTION 

Discipline-related student organizations have long been 
valued by Schools and Collegesof Business as a vehicle 
to broaden student experiences. Student chapters of 
national professional oiganizations in accounting 
(Institute of Management Accountants), operations 
management (American Production Inventory Control 
Society), marketing (American Marketing Association), 
and many others thrive on college campuses around the 
country. However, there does not appear to be a like 
professionalorganization in Information Systems with a 
significant number of student chapters. 



Although there may not be a national umbrella 
professionalorganization with which many M IS student 
organizations have chosen to affiliate, our research 
indicatesthat many MIS programs have establishedMIS 
student organizations. Because these organizations are 
largely unaffiliated, there is little standardization from 
organization to organization. Each MIS student 
organization, therefore, chooses which activities it 
undertakes, the level of formal structure of the 
organization, and the types of goals it pursues. 
Undoubtedly, some organizations are more successful 
than others. We postulate that an understanding of the 
factors that contribute to successful organizations can 
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lead to a framework for an organizational structure and 
an activities set that will lead to improved MIS student 
organization effectiveness. 

The purpose o f this research is to determine a framework 
for success in MIS student organizations and to identify 
those factors which can most positively contribute to that 
success. We examine the structures and activities of 
successful MIS student organizations to identify best 
practices. These best practices are assembled into a 
comprehensive organizationalstructure andactivitiesset 
that lends standardization to the organization and 
improves organizational effectiveness. 

REVIEW OF LITERATURE 

A review of the literature on student organizations and 
learning was performed to determine what is formally 
known about the relationship between student 
organization activities and student learning. Educators 
have proposed a num ber of learning models, including 
collaborative learning, cognitive informationprocessing, 
and sociocultural learning [Leidner and Jarvenpaa, 
1995]. These learning models are not necessarily 
exclusive they can be alternated or offered in 
conjunction with one another. It would appearthat well- 
grounded student organization can promote learning 
through each of these models. A student IS organization 
performs an important function to a successful IS 
program. Student organizations are a valuable 
supplement to the learning a student acquires with the 
standard IS curriculum. A MIS studentorganization can 
promote, among other things, learning about current IS 
practices and professional ethics. 

Money (1994) has proposed that universities should 
explore new channels of product delivery off-site 
courses, e-education, etc. However, the research article 
also calls for broadening students’ educational 
experience and speaks of the “content-based dangers of 
IS curriculum because of limited linkages between 
Universitiesand practitioners’ worlds (p.50), citing the 
work of Spruell and Franz (1993). Argyris (1990) 
theorizes that support for the educational experience of 
the student is needed beyond the classroom and program 
of course work. Each of these papers supports the 
broadening of educational experience through 
cooperative experiences, internships, field trips, guest 
speakers, and, by implication, student organizations. 



Cale (1994) provides an example of a very intense 
broadening and deepening linkage between the 
University and practitioner world through studentwork 
with a mentor company an activity that could be 
supported through a MIS student organization. Fry 
(1993) also makes a case for tight linkage between 
education and industry. This work describes successes 
from a cooperative curriculum development program 
partnering academics and industry representatives. An 
industry advisory board for a MIS student organization 
would appear to be in sync with this approach. Jacobs 
(1995) discusses the role of an Advisory Board and how 
it can tighten linkages between academics and 
university including providing speakers to students. 

Pierce and Henry (1994) found a difference between 
how educators and IS professionals perceive ethical 
issues, practices, abuse opportunities, extent of abuse, 
etc. One example of these differences is that 

professionals see ethical issues arising more frequently 
than do academics (i.e., more of a real world problem). 
Because of these differences, educators may not be 
teaching ethics as knowledgeably as it could be taught. 
The authors conclude that, since teaching ethics is not 
easy (especially given that academics and IS 
practitioners have different perceptions), it is important 
that educators and professionals form a partnership. 
Student organizations can provide an opportunity to 
foster th is partne rship. 

Dick, et al. (1995) suggest colloquium evenings and 
short seminars as appropriate vehicles to deliver an 
understanding of professional ethical expectations, the 
role of IS professionals, negotiation skills, and where an 
IS career might lead. Benbasat et al. (1980) have noted 
thattoo much emphasis has traditionally been placed on 
developing students’ technical skills and not enough on 
behavioral or organization skills. The aforementioned 
colloquia and short seminars areone means of increasing 
behavioral and organizational skills. Student 
organizations would appear to be an ideal platform for 
reinforcing students’ understanding of these critical 
skills. 

Dick, et al. (1995) have reviewed the literature on 
learningstrategies, including action learning, workplace 
learning, etc. Hughes and McLean (1992) have surveyed 
literature on many of these learning strategies. Dick sees 
workplace learning as especially relevant to cooperative 
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education. Dick notes that there is an important 
distinction between workplace learning and workplace 
training: workplace learning emphasizes organizational 
and behavioral skills, and that the emphasis is on the 
learner rather than the teacher. Many student 
organization activities, if approached correctly, can lead 
to these types oflearning. 

One informal method of workplace learning is dialogic 
learning learning through informal discussions with 
other workers. Gardiner & Singh (91) have found that 
cooperative education program s in Science and IS result 
in students whose transition to the workplace is less 
difficult than their (non-coop) peers. Cooperative 
education is a strategy of applied learning which is 
structural, developed, and supervised by a university in 
collaborationwith one ormore employing organizations. 
The philosophy underlying cooperative education is 
closely aligned with the experiential learning strategy 
(Galliers, 1987; Lee 1987; Little and Magetson, 1989; 
and Wersky, 1989). Sm all, et al. (1998) have found that 
cooperatives can provide another avenue for students to 
engage in experiential learning, including through 
community service projects. While lacking the depth of 
cooperativeeducation, student organization professional 
activities can increase the breadth of experiential 
activities. 

Pierce and Henry (1994) have concluded that it is 
important to instill ethical standards in IS students. A 
role of the MIS student organization can be to promote 
professional ethics. 

Romm and Wong (1997) have found that the best 
method to teach softskills is through case study. Cragg 
(1998) found that site visits serve as a form of live case 
study on IS courses. The site visit brings reality to the 
students and provides variety in the learning experience 
while encouraging action learning. Further, the author 
contends thata site visitallows for a rich opportunity for 
students to explore theory and practice, expose students 
with little experience to the complexities of 
organizations, and help educato rs to keep up to date with 
current practices and problems. Many student 
organizations organize visits to industry sites. 

Summary 

We found a substantial body of literature indicating that 
many of the activities that student organizations can 
undertake have a positive effect on student learning. It 
would appear that activitiessuch as seminars, colloquia. 



speaker visits, visits to industry sites, promotion of 
professional ethical standards, and collaboration with 
advisory boards enrich and broaden the student learning 
experience. This study attempts to determine the scope 
and frequency of such activities in today's successful 
MIS student organizations so as toestablish a framework 
of best practices. Such a framework should allow 
student organizations from across the country to assess 
the effectiveness of their organization and to identify 
additional activities that the organization can pursue to 
be more successful. 

METHOD 

This study was carried out in phases. First, the literature 
on student organizations and learning was reviewed to 
determ ine what is form ally known about the relationship 
between student organizations and student learning. 
Second, a search of the Internet was conducted in an 
attempt to find a representative sample of convenience 
ofMIS student organizations (see Appendix A). Thirty 
organizations were identified and included in the study. 

In the third phase of the study, representatives (student 
officers or faculty advisors) from these organizations 
were contacted via email and asked to respond to a one- 
page survey (Appendix B) on organizational 
effectiveness. The representatives were given a choice 
of responding either through a website devoted to 
collection of the survey responses or by email reply. 
The survey instrument is adapted from the Doty, et al. 
seminal research on organizational effectiveness 
measures and has been shown to provide a reliable and 
valid metric. The purpose of the survey was to 
determine which of the sample organizations are most 
effective. 

Representatives from thirteen organizations (43%) 
responded to our email request. Once the survey 
responses were received, each organization’s score was 
tabulated. The highest-scoring responders were 
consideredthe most successfulstudentlS organizations. 
Once the most successful MIS student organizations 
were identified from the sample, the web sites of these 
organizations were content analyzed to determine their 
organizationalstructure and to identify the activities that 
are conducted by the organization. 

The resulting analysis produced a listing of stmctural 
components and activity components, which represents 
the collective structural types and activities set of the 
successful organizations. From these lists, the 
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researchers constructed a framework for the model 
organization that is consistent with current theory on 
extracurricular learning as identified in the literature 
review. 

RESULTS 

In response to the e-mail solicitation, completed surveys 
were received from thirteen organizations. These have 
been coded and organized by total scores, which are 
summarized at the top of the Table 1 . The leftmost case 
had the highest score and the rightmost, the lowest. 



Individualitem responses are listed byscale anditem ID. 
Questions 4 through 7, not being derived from a 
previously validated scale, were not used in computing 
rankings. Future studies with larger samples may 
examine correlation o f these item s with validated scales. 

Content analysis of the six highest rated organization’s 
websites is summarized in Table 2. The features most 
commonly found at the highly ranked student 
organizations’ websites are listed toward the top of the 
chart. 



TABLE 1 

SURVEY RESPONSE DATA 



Case # 


Cl 


C2 


C3 


C4 


C5 


C6 


C7 


C8 


C9 


CIO 


Cll 


C12 


C13 


Years 


2 


4 


1 


2 


1 


1 


2 


0 


2 


2 


1 


5 


3 


Scalel 


4.4 


4 


4.2 


3.8 


4 


4 


3.6 


3.6 


3.6 


3.4 


3 


2.8 


2.4 


Scale2 


4.4 


5 


4.2 


4.2 


3.6 


3.6 


3.4 


3.4 


3.4 


3 


3.2 


2.4 


2.2 


Scale3 


5 


5 


5 


0 


4 


3.6 


0 


0 


0 


3.6 


0 


0 


2.2 


Wt_Avg4.55 


4.5 


4.4 


4 


3.9 


3.8 


3.5 


3.5 


3.5 


3.35 


3.1 


2.6 


2.3 


Z Score 


0.138 


0.125 


0.100 


0.000 


-0.025 


-0.050 


-0.125 


-0.125 


-0.125 


-0.163 


-0.225 


-0.350 


-0.425 


Rank 


1 


2 


3 


4 


5 


6 


9 


8 


7 


10 


11 


12 


13 


Ql-a 


3 


5 


4 


3 


4 


4 


4 


3 


3 


4 


2 


3 


2 


Ql-b 


5 


4 


5 


4 


4 


5 


4 


4 


4 


4 


5 


4 


3 


Ql-c 


4 


3 


4 


3 


3 


3 


3 


3 


3 


3 


2 


2 


2 


Ql-d 


5 


4 


4 


4 


4 


4 


4 


4 


4 


3 


3 


3 


2 


Ql-e 


5 


4 


4 


5 


5 


4 


3 


4 


4 


3 


3 


2 


3 


Q2-a 


3 


5 


4 


5 


4 


4 


4 


3 


3 


3 


3 


3 


3 


Q2-b 


5 


5 


5 


5 


3 


4 


4 


4 


4 


3 


5 


4 


2 


Q2-C 


4 


5 


4 


3 


3 


3 


3 


3 


3 


3 


3 


2 


2 


Q2-d 


5 


5 


4 


4 


3 


4 


4 


4 


4 


3 


3 


3 


2 


Q2-e 


5 


5 


4 


4 


5 


3 


2 


3 


3 


3 


2 


0 


2 


Q3-a 


5 


5 


5 


0 


6 


4 


0 


0 


0 


4 


0 


0 


3 


Q3-b 


5 


5 


5 


0 


3 


4 


0 


0 


0 


4 


0 


0 


2 


Q3-C 


5 


5 


5 


0 


3 


3 


0 


0 


0 


3 


0 


0 


2 


Q3-d 


5 


5 


5 


0 


3 


4 


0 


0 


0 


3 


0 


0 


2 


Q3-e 


5 


5 


5 


0 


5 


3 


0 


0 


0 


4 


0 


0 


2 


Q4 


5 


3 


5 


5 


5 


4 


4 


4 


4 


4 


4 


4 


1 


Q5 


5 


3 


4 


4 


4 


2 


4 


3 


4 


5 


5 


4 


1 


Q6 


5 


5 


5 


4 


5 


4 


5 


4 


3 


5 


5 


4 


1 


Q7 


5 


3 


5 


3 


3 


4 


4 


3 


3 


3 


4 


4 


1 


Natorg* 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


N 


Y 


Y 


N 


Y 


Y 
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TABLE 2 

ACTIVITIES OF THE HIGHEST-RATED ORGANIZATIONS 



Case Number-> 


1 


2 


3 


4 


5 


6 


Feature (Instances) 














Mission Statement (6) 


X 


X 


X 


X 


X 


X 


Link to University Career Center (5) 


X 




X 


X 


X 


X 


Event Calendar (5) 


X 




X 


X 


X 


X 


Officer / Committee Listings (4) 


X 


X 


X 


X 


X 




Job Listings (4) 


X 




X 




X 


X 


On Line Membership Application (4) 


X 


X 




X 


X 




Social Event Announcements (4) 


X 




X 




X 


X 


Corporate Sponsor Acknowledgement / Links (3) 


X 




X 


X 






Link to MIS Department (3) 


X 




X 


X 






Current News or “What’s New (3) 


X 




X 


X 






On-Line Resumes (3) 


X 




X 


X 






Links Page General Interest (3) 






X 


X 




X 


Links to Global (non- university specific) Job Search Tools (3) 








X 






Links to Tutorials and Research Aids (3) 




X 


X 


X 






Automated e-mails to Members (3) 








X 


X 


X 


Speakers from Off-Campus (3) 


X 




X 




X 




Alumnae Database / Links (2) 


X 




X 








Internal Discussion Groups/ Bulletin Boards (2) 


X 








X 




Links to Other Campus Resources (2) 






X 


X 






Links to University News (2) 








X 




X 


Multimedia Intro (2) 






X 




X 




Links to Affiliate Sites (2) 










X 


X 


Speakers from Campus (2) 






X 




X 




Trips / Site Visits (2) 


X 




X 








Recruiter Postings-Externally Updateable (1) 


X 












Links to Member Home Pages (1) 




X 










On-Line Help for Using Site (1) 




X 










Meeting Minutes (1) 








X 






Activity Pictures (1) 










X 




President’s Message (1) 










X 




Interest Groups / Learning Groups (1) 










X 




Constitution /Bylaws (1) / 






X 









CONCLUSIONS 

The results from the study indicate that the most 
effectivelS student organizations carry outactivitiesthat 
are consistent with the student learning activities that 
have been identified in our literature review. Many of 
these relate to future employment opportunities and are 
intended to foster closer contact between the student 
members and business and industry. Each of the 
successful organizations has a mission statement. Most 



have an active relationship with the institution’s Career 
Center and keep student members informed of upcoming 
activities through an online calendar. While most 
organizations publicized their officer’s contact 
information, each organization offered an entirely 
different set of club-related activities. 

Based upon the sample results, we recommend that a 
student IS organization consider implementing the 
following structural components: 
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Mission Statement; 

Online link to University Career Center; 

Online event calendar; 

Online list of Officers and committees; 

Online membership application; and 
Online social event announcements. 

In addition, we recommend that a student IS 
organization plan, organize, and carry out many 
activities each year. These activities include: 

Develop and post an online resume book; 

Provide links to general and job-specific search 
tools; 

Provide links to tutorials and research aids; 

Provide automated email to members through the 
organization website; 

Host speakers from off-campus. 

Each of the most successful student IS organizations 
identified as part of this study had an extensive and 
robust activities set that serves to keep members 
involved and enthused. In addition, each organization 
appeared to have made a substantive effort to open 
communication channels amongst its members and with 
the larger business community. These philosophies, 
along with the structural and activities components 
suggested here, should improve the effectiveness of any 
IS student organization. 
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APPENDIX A 

RESULTS FROM WEB SITE SEARCH 
These URLs indicate the web sites that were identified and used in the search: 



http://www.mis.iastate.edu/ 
http://misclub.hawaii.edu/ 
http://www.cob.sjsu.edu/assoc/misclub/ 
http://www.cob.sjsu.edu/misclub/ 
http://www.biz.uiowa.edu/studorgs/misa/ 
http://snail.bpa.arizona.edu/-misga/ 
http://www.cba.uga.edu/smis/ 
http://www.uidaho.edu/-misclub/ 
http://www.hbg.psu.edu/clubs/mis/ 
http://www.uwosh.edu/home_pages/colleges/coba/ 
Academ ics/majo rs/mis/m is-club.htm 
http://octavius.coba.uwosh.edu/mis_club/ 
http://www.rowan.edu/mars/clubs/mis/ 
http://sbe.d. umn.edu/misclub/ 

http://www.bus.orst.edu/students/std_org/sim/what.htm 

http://www.uwec.edu/Student/SIM/ 



http://www.sacmisa.com/home/ 

http://missadhs.org/ 

http://www.clubs.psu.edu/misclub/ 

http://www.sba.oakland.edu/misclub/ 

http://www.udayton.edu/-misclub/ 

http://www.slu.edu/organizations/mis/ 

http://www.umsl.edU/business/mis/mis_club.html.l 

http://www.umsl.edu/studentlife/misclub/ 

http://www-cob.csuchico.edu/missociety/spring2000/ 

http://www.csun.edu/misa/ 

http://www.cob.fsu.edu/students/orgs/mis/ 

http://www.uwm.edu/StudentOrg/AITP/ 

http://mis.njit.edu/ 

http://www.bu s.ucf.edu/cgi -bin/redirect/redirect. 
pl?http:/Av ww.bus.ucf.edu/misa/ 
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APPENDIX B 

IS STUDENT ORGANIZATION SURVEY 



We areconducting research into student organizations in the Information Systems field. Please help us conductthe 
research by filling outthe following survey - either by reply email (the survey is enclosed below) or by going directly 
to the research survey web site at: http://misntindstate.edu/isorg.htm Please answer each of the questions/items to 
the best of your ability. Thank you for your assistance! 



Your organization’s name: College/University: 

Your position with the organization: President Faculty Sponsor Other(describe) 

How long have you been associated with the organization? years 



1 . Compared to what you would like it to be, the. . . 

a. time and effort required of the membership to produce your 

organization’s services to students is 

b. quality of your organization’s services to students is 

c. involvement per member in your organization is 

d. satisfaction of most members with the organization is 

e. effectiveness of communications among your organization’s 

committees and the membership is 



Low Medium High 

4— 5— 6— 7 
3-~ 4— 5— 6— 7 

1.. .2-3— 4— 5— 6— 7 
l___2-.-3— 4— 5— 6— 7 

1 .. . 2 — 3 -‘ 4 -> 5 — 6— 7 



2. Compared to other student organizations at your school, the. . . 

a. time and effort required of the membership to produce your 

organization’s services to students is 

b. quality of your organization’s services to students is 

c. involvement per member in your organization is 

d. satisfaction of most members with the organization is 

f. effectiveness of communications among your organization’s 
committees and the membership is 



Low Medium High 



l___2.-3— 4 — 7 
1___2— 3— 4 ->- 5 -- 6 — 7 
1__.2->3— 4 — 5 — 6 — 7 
l-_-2— 3 - 4 — 5 — 6---7 

1 ..- 2 .- 3 - 4 — 5 — 6— 7 



3. Compared to IS student organizations at other schools, the. . . 

a. lime and effort required of the membership to produce your 

organization’s services to students is 

b. quality of your organization's services to students is 

c. involvement per member in your organization is 

d. satisfaction of most members with the organization is 

g. effectiveness of communications among your organization's 
committees and the membership is 



Low Medium High 

1__.2— 3— 4— 5— 6— 7 
1__.2^_3— 4— 5— 6— 7 
1-..2— 3— 4— 5— 6— 7 
1_..2-3— 4— 5-6— 7 

1^-2— 3— 4— 5— 6— 7 



Don't 

Know 

9 

9 

9 

9 

9 

Don’t 

Know 

9 

9 

9 

9 

9 

Don’t 

Know 

9 

9 

9 

9 

9 



Rate your agreement to the following: 

Strongly Strongly 

Disagree Neutral Agree 

Our student organization does a very good job in reaching our goals. 1 — 2 — 3 — 4 — 5 — 6 — 7 

Our student organization has grown at the same rate or faster than the rate of new 

students in the IS area at our school. 1 — 2 — 3 — 4 — 5 — 6 — 7 

Our organization has promoted student learning in the field of IS. 1 — 2 — 3 — 4—5 — 6 — 7 

Our organization is seen as one of the premier organizations on campus. 1 — 2 — 3 — 4 — 5 — 6 — 7 

Approximately, how many members currently belong to your organization? members 

Would you like to see your student organization become a local chapter of a national student MIS organization? Y N 



Please use the space below to provide us with any other comments you would care to make: 
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The expected "Dis-lnter-Mediation" of electronic markets has not yet occurred. To the contrary, E-business 
technologies have created a "Re-Inter-Mediation of Markets", Whether a least-cost-driven management failure or 
a significant software deficiency, this current reality needs to be critically assessed. 

The biggest current impediment to E-business is that IT-connectivity is remarkably "dumb" about recognizing 
context — Web searches result in unwanted and overloaded information, A paradigm shift in the development of the 
information highway calls for "clever" connectivity overcoming current limitations of hypertext and other Internet 
make-up languages. 

Whether such cleverness is a software issue or a result of the growth of a new info-mediary industry, is an issue of 
some conjecture. This paper explores the emerging contours of an info-mediary industry and suggests that much is 
to be done in order to render IT-connectivity genuinely clever, genuinely inter-connected - capacities crucial to an 
effective knowledge-based economy. 



INTRODUCTION 

In the era of widespread economic and technological 
change, understanding the changing nature of work is 
important to understanding organization and re- 
organization (Barley, 1996). The adoption of new IT 
also conveys a powerful cultural load, having the 
capacity to involve all organizational actors in its use 
and being inserted into organizational life in both 



material and discursive ways (Webster and Robins, 
1986; Hill, 1988; Muetzelfeldt, 1988; Korac-Boisvert, 
1992). Materially, IT provides the potential for a wide 
range of data collection, storage and processing. IT 
provides information on demand, builds banks of shared 
knowledge and enables real-time, structured learning 
events to transcend boundaries of time and space - 
becoming a tool for building solutions (McAteer, 1 994: 
68 ). 
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IT's discursive presence is most prominent in a cultural 
setting concerned with the future of work and 
organization (Forester, 1985; 1989; Roszak, 1986; 
Webster and Robins, 19 86). Within a sociological 
context, IT influences social system continuity; control; 
identity; and the integration of members. For example, 
the micro-electronic advancements of the last decade 
have been particularly influential on labour 
organizations. The use of job clustering is often 
facilitated by the use of IT, as opposed to job 
simplification, emphasized by the Taylorist influence 
(Hammer and Champy, 1993: 51-52). 

Furthermore, when assisted by direct accessto centrally- 
formed databanks, by means of computer networks and 
other forms of inter-active IT, employees at the lower 
levels of an organizational structure have the potential to 
become multi-skilled; equipped with the kind of 
planning and logistical information hitherto held only at 
the disposal of middle-managers. The distribution of the 
power within the work force, however, is not pre- 
determined (Child, 1987). 

During the 1980s, the cost of IT's material components 
(hardware) continued to decline (Kauffman and Weill, 
1990), resulting in IT permeating every facet of an 
organization and, concurrently, becoming available for 
individual use. In the 1990s and beyond, IT further 
intensified its dominant role by the ever-increasing 
societal dependence on IT systems that has segmented 
the labour market into three generic groups: "routine 
production servers", "in-person servers" and "symbolic 
analysts" (Reich, 1993). The proliferation of IT has 
further re-defined traditional routine production work 
into sequences of repetitive tasks, to the extent that even 
the supervisors of such tasks are easily replaced. There 
is, currently, no shortage of such labour and it usually 
can be found more cheaply in a new market (Reich, 
1993). Routine production margins are controlled, 
profits are usually predictable and workers have a high 
degree of exposure to global competitive forces. 

CONNECTIVITY OR INFORMATION REACH 

Whilst, in the past, information components ofthe value 
chain have been deeply embedded in the physical value 
chain, because the information delivered from sender to 
receiver was carrie d physica lly by post, E-m ail or orally 
by the person (face-to-face inter-action or telephone), it 
reached target audiences only. Information was 
constrained to follow the linear flow of the physical 
value chain (Evans and Wurster, 1997). This physical 
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chain of communication media is an invariant property 
of communication "richness" or "leanness" (Daft and 
Lengel, 1986). However, with the advancement of IT, 
there is potential for everyone to be connected 
electronically, enabling communication rich 
information to travel by itself. This, in turn, makes it 
possible to un-bundle information from its physical 
carrier (Evans and W urster, 1997). 

Connectivity, or "information reach", is currently limited 
to the Internet, which connects everyone having access 
to required technology; extra-nets, which connect 
organizations to one another and intranets, which 
connect individuals within organizations (Evans and 
Wurster, 1997). However, the quality of information or 
"information richness", defined as 'the ability of 
information to change understanding within a time 
interval' (Daft and Lengel, 19 86: 554), an d characte rized 
by three attributes, is not necessarily compatible with 
connectivity or "information reach" (Evans and Wurster, 
1997): 

The bandwidth, or the amount of information, that 
can be moved from sender to receiver in a given 
time (text is narrowband; a multimedia is 
broadband). 

The degree to which information can be customized 
(TV advertising is less customized than personalized 
mail, but reaches more people). 

The inter-activity (dialogue accommodates exchange 
between a limited number of people but the 
monologue can accommodate exchange between one 
and many). 

Although communication media (face-to-face, telephone, 
E-mail and documents) vary in the capacity to reach and 
process rich information, there is, currently, no single 
medium which can achieve both information "richness" 
and "reach". Some software packages, such as Quicken, 
Money and proprietary bank software, permit rich 
exchange but only with the customer's own bank (Evans 
and Wurster, 1997). Web "browsers" have the potential 
to reach the entire universe of financial institutions 
(Evans and Wurster, 1997). However, face-to-faceis the 
richest medium because it provides immediate feed-back 
so that interpretation can be checked and 'multiple cues, 
via body language and tone of voice and message 
content, are expressed in natural language intervals' 
(Daft and Lengel, 1986: 560). 
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Information, in many ways, defines pair-wise relations, 
such as buyer-seller relationships, where, traditionally, 
much of the trader's ma rgin depended on the asymm etry 
of information (Evans and Wurster, 1 997). For example, 
in trade, the caveat emptor applies and the buyer of 
goods or services must lookout for his/her own interests. 
Thus, a merchant is permitted to negotiate the best deal 
he/she can get and need not consider what is in the best 
interest of the customer. Thus, increasingly, customers 
value mediums which are both rich and reachable, as 
he/she will need information that is complete (what was 
conveyed), truthful (authenticity of conveyed 
information), clear (what has been conveyed) and 
contextual (establish context of information origin) 
(Ngwenyama and Lee, 1997). This validation of 
information pertainingto the completeness, truthfulness, 
clarity and contextually, and the sheer breath of choices 
of medium and databases available to potential 
customers, will require services of an inter-mediator or 
a "symbolic analyst" (Reich, 1993) a qualified actor 
who can access, analyze and synthesize information, can 
add to the value chain or produce "symbolic goods" 
(with the focus on intellectual fields) (Bourdieu, 1971; 
1979), shape conditions of supply and demand for 
symbolic goods (the process of competition and 
monopolization) and mediate the struggle between the 
established and outsiders. 

For example, some organizations have an incentive to 
create, or simply make available, databases on interest 
rates, risk ratings and service quality histories. New 
opportunities will emerge for third parties that neither 
produce a product nor deliver a primary service inter- 
mediators (Evans and Wurster, 1997). Navigators or 
agent brands have been around for long time. For 
example, restaurant guides influence readers towards a 
particular establishment. The Platform for Internet 
Content Selection (PICS) is aprogramming standard that 
allows net browsers to interpret third-party rating labels 
on Web sites. PICS en ables users to rate anythin g and it 
makes those ratings ubiquitous, serviceable, sortable and 
costless (Evans and Wurster, 1997). The dramatic 
proliferation of net-wired matters increases the need for 
such navigators and other facilitating agents; those that 
guarantee a product's performance or assumed risk 
(Evans and Wurster, 1997). 

Ethnographic evidence suggests that competition to 
acquire symbolic goods in the information class 
generates high admission barriers and effective 
techniques of exclusion (Douglas and Isherwood, 1980: 
180; Industry Commission, 1995). As well, there is the 



newcomer's perceived need to adopt subversive 
strategies in order to create a space for themselves and 
displace the established. This is the very process of de- 
monopolization and de-hie rarchization of previously 
established and (il)legitimate cultural enclaves which 
had brought about a phase of cultural de-classification in 
society and global shifts (DiMaggio, 1987). Cultural 
shifts need to be comprehended in terms of how they 
inter-relate to intra-societal, inter-societal and global 
processes. 

CONNECTIVITY AND INTER-ACTIVITY 

Notwithstanding that inter-activity is often perceived as 
a characteristic of dialogue, in reality, it is not limited to 
un-mediated communication. Rather, inter-activity can 
be understood as a formal element of both un-mediated 
and mediated communication. Inter-activity can be 
conceptualizedas aresponsiveness in inter-personal and 
societal comm unication with a message's thread or chain 
of inter-related messages (Rogers, 1986; Rafaeli, 1988; 
Rafaeli and Sudweeks, 1997). Furthermore, the degree 
to which communication transcends reaction is crucial 
(Rafaeli, 1988). Thus, one-way communication is not 
inter-active as there is one source that sets the agenda, 
receiving no or, in some cases, indirect feedback. 
Moreover, in two-way and re-active communication, 
where both sides "send" messages but where there is no 
feedback, there is only potential for inter-activity - two- 
way communication is present as soon as messages flow 
bilaterally. Re-active settings require, in addition, that 
later messages refer to (or cohere with) earlier ones 
(Rafaeli, 1988:1 19) before they become inter-active. 

At its minimum, effective communication requires one 
re-active thread, which may reach a fully inter-active 
level (Rafaeli, 1988). The two-way communication 
remains re-active unless latermessages in any sequence 
take into account not only the messages that preceded 
them, but also the manner in which previous messages 
were re-active (Rafaeli and Sudweeks, 1997). Only then 
is a fully inter-active level reached, implying that 
communication roles in inter-active communication are 
inter-changeable (Rogers, 1995: 314). 

Inter-activity is often defined in its wider use to include 
a variety of meanings (Goertz, 1995;Fredin and David, 
1998; McMillan and Downes, 1998) - being "connected" 
via the "Net" does not necessarily imply inter-activity 
(Noth, 1996). Electronic media can neither produce nor 
share meaning in a narrow sense but can only facilitate 
or impede inter-active communication. The reality is 
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that even episodes of face-to-face communication, 
regardless of whether they are mediated or not, are 
seldom fully inter-active, although many aspire towards 
full inter-activity as it often implies equality of 
participation (Rafaeli, 1988). Inter-activity, in this 
narrow sense, describes and prescribes the manner in 
which conversational inter-action, as an iterativeprocess, 
leads to jointly-produced meaning (Rafaeli and 
Sudweeks, 1997). However, it had been argued that 
different purposes and tasks require different 
communication settings and different levels of inter- 
activity (Hoflich, 1996). 

IT-ENABLED COMMUNICATIONS AND 
ORGANIZATIONAL CONNECTIVITY 

IT connectivity and communication studies are published 
across many disciplines such as IT; linguistics; 
psychology; management; business and sociology 
inducing the rapid proliferation of research. In order to 
operate effectively in a changing environment, 
organizations need to process information for the 
purpose of reducing uncertainty (Galbraith, 1973b) and 
resolving equivocation (Weick, 1979; Daft and Lengel, 
1984; 1986). Organizations need to provide connectivity 
between organizational actors and their client base. 
Communication underliesmost organizationalprocesses 
and may, therefore, be considered the essence of 
organizing as organizationalmembers inter-actinternally 
and externally to co-ordinate activities, disseminate 
information and make decisions (Galbraith, 1973a; 
Tushman and Nadler, 1978; Euske and Roberts, 1987; 
Weick, 1987). 

Effectivedecision-making is dependent upon processing 
information in an internal and external organizational 
environment, which continuously changes (Weick, 
1987), by permitting horizontal and verticalinformation 
flows. Vertical information processing is used to 
interpret the environment and to reduce uncertainty 
whilst horizontal information processing serves to co- 
ordinate internal activities (Daft and Lengel, 1984). In 
contemporary organizations, IT-mediated communi- 
cation is seen as a solution for effective horizontal and 
vertical comm unication. IT-enabled comm unication is 
an a-synchronous, computer-mediated message system 
that uses computer text processing and communication 
tools to provide high-speed information exchange 
(Sproull andKiesler, 1992). IT-enabled communication 
media have been defined to include electronic mail; 
voice mail; computer bulletin boards; computer 
conferencing; Group-Ware; group-decision support 

146 



systems, as well as other new forms of structured 
communication support via computers (Steinfield, 1992: 
349). 

The distinguishing characteristics of IT-based 
communication media have been developed around the 
categories of synchronicity; transmission speed; textual 
and graphical nature of the medium; its multiplicity of 
connections; and its storage and manipulation facilities 
(Garton and Wellman, 1995). Where the medium is 
more than the carrier of the message, 'the medium itself 
becomes the message' (Trevino, Daft andLengel, 1990: 
84). That is, effective communication channels need to 
have the capacity to transmit data and meaning. 
McLuhan (1964) submittedthe original proposition in its 
most extreme form in claiming that "the medium is the 
message". Later, more moderate versions suggestedthat 
the 'medium is also a message' (Garton and Wellman, 
1995: 438). Media are also symbols of something and 
this is defined in the particular context of the 
organization (Sitkin, Sutcliffe and Barrios-Choplin, 
1992). 

For example, communicationtechnologies have specific 
characteristics that condition their impact and use 
(Culnan and Markus, 1987; Finholt and Sproull, 1990). 
Electronic mail is a-synchronous, fast, text-based, one- 
to-one, one-to-m any or many-to many. These attributes 
make new communication activities possible as well as 
restricting communicative possibilities of conventional 
channels. As such, electronic mail alters patterns of 
communication in increasingthe vertical and horizontal 
communication flow through the organization. For 
example, a-synchronous media, such as electronic mail 
and voice mail, are able to remove time zones and 
geographical barriers and to distribute messages to a 
large number of individuals simultaneously and rapidly. 
This has been seen as a core advantage of an a- 
synchronous technology. Simultaneous distribution of 
information is perceived to enhance an actor's 
connectivity by more effective co-ordination of project 
groups, scheduling ofmeetings and general information 
exchange. In terms of scope, electronic mail increases 
the total number of participants in the message system, 
the number of inter-connections between them and the 
number ofmessages sentand received. In increasing the 
total number of communication participants, IT 
facilitates the establishment of new contacts within the 
organization. 

At the organizational level, electronic mail is seen as a 
device for improving effectiveness in expanding the 
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range of information and expertise and facilitating the 
exchange across the organization and its decisionmakers 
(Adams, Todd and Nelson, 1993). Electronic mail is 
also preferred for communicating negative news in order 
to avoid open dialogue and conflict resolution (Sproull 
and Kiesler, 1986). 

CONNECTIVITY AND 
INFORMATION NETWORKS 

According to Katz and Kahn (1996), the act of 
organizing occurs through communication, whereby 
information is exchanged and meaning fabricated 
(Orlikowski and Yates, 1994). This view holds that 
communication is central to the organization process. 
As such, organizations can be viewed as communication 
systems in which networks of nodes (individuals, 
groups, departments) are linked with one another in a 
variety of ways and linked with elements in their 
external environments. Weick(1987: 97-98) has argued 
that inter-personal communication is the 'essence of 
organizationbecause it creates stmctures that then affect 
what else gets said or done and by whom. The 
structures, in turn, create additional resources for 
communication, such as hierarchical level, common 
tasks, exchangeable commodities and negotiable 
dependencies'. 

In reality, all organizations have an information network, 
although they have been variously named by scholars 
and practitioners alike such as "communication 
network" or an "information processing network" 
(Rubber, 1989). Whatever the label and whatever the 
mediate used, their transactional content is information. 
They exist to transfer information from those whose job 
it is to acquire, store and process information to those 
who must use the information to make a decision. 

The information network canbe formal (prescribed) and 
informal (emergent) information flows in an 
organization. Information, especially through the 
network, is an integral part of an organization; it is the 
essence of any organization affecting structure through 
the political process. As information flows through the 
network, it is changed in a number of ways. It is likely 
to be aggregated. Its meaning may change through 
interpretation at each node. Information is shaped by 
particular fonus of knowledge when it is created and 
shaped again by other knowledge when it is received and 
interpreted. This information does not transfer an 
objective core of knowledge about the world (the 
positive interpretation of information) when it is "read" 



by the recipient. Rather, it enters into a culturally 
mediated, but under-determined, relationship to the 
knowledge of the recipient. Organizational members are 
not homogeneous; different individuals have differing 
behavioural and personal factors. Interpretation of 
information is very subjective and dependent not only on 
task complexity but also on decision-making 
characteristics. Cognitive style refers to differences that 
can be perceived in individuals due to differing cognitive 
structures. Mason and Mitroff (1973) have suggested 
that each person possesses a particular specific 
psychological cognitive style or "personality" and that 
each personality type utilizes information in different 
ways. 

Meaning can never be literally transferred from one 
person to another all that passes between message 
sender and message receiver is a pattern of physical 
stimuli. Computer-mediated communication has the 
ability to change how information is received and 
distributed (Kiesler, 1986). This usually results in a 
fundamental change in the information that is made 
available to organizational members and the source of 
that information. Nodes on the information network can 
be by-passed, whilst having access to an information 
source may reduce the negative consequences ofvarying 
interpretations. 

ELECTRONIC GROUPS, NETWORKED AND 
VIRTUAL ORGANIZATIONAL DESIGNS 

In contrast to conventional groups, electronic groups are 
supported by IT-enabled communication media such as 
electronic mail and Group-Ware systems (Lotus Notes; 
Microsoft Exchange). The Group-Ware technology 
allows individuals to form groups foremerging purposes 
that cut across geographic and departmental boundaries. 
Group-Ware substantially facilitates information 
exchange and decision-making in groupsindependent of 
physical location. Whilst Group-Ware technologies 
might produce efficiency gains, theymay, alternatively, 
create new or different forms of groups and, as such, 
induce un-desirable organizational consequences, as 
management policies strongly influence the nature of 
electronic groups (F inholt and Sproull, 1 990). Thus, the 
activities and the ratio of positive and negative 
consequences for the oiganization through 

attributes of group behaviour (a physical setting, 
member characteristics, membership criteria, task 
type); 



Proceedings of the 15^ Annual Conference of the International Academy for Information Management 



147 



processes of group behaviou r (inter-action, influence 
attempts, identity maintenance); and 

organizational consequences of group behaviour 
(participation, performance, learning) 

are all modified by organizational polices and 
organizational actors. Media usage left to discretion and 
self-policing may account for negative effects such as 
information over-load or abuse (Finholt and Sproull, 
1990). Furthermore, technical policies may include the 
installation of filters and group directories. At the social 
level, norms and codes for use should be endorsed to 
encourage responsible and effective technology use. 

New designs and structures which have been depicted as 
the breakdown of hierarchy are enabled through the 
emergence of IT-medi ate d communication (Marlow and 
Wilson, 1997). The inter-active, multi-level capabilities 
of communication technologies compel change in 
conventional organizational activities and designs and, 
as such, drive the emergence of new organizational 
forms, exemplified by the network or the virtual 
organization. Their value has been specifically 
identified in relation to the organization's inter-actions 
with external constituencies. The new organizations 
capture value in shifting from the traditional "make and 
sell" to the new IT-enabled paradigm of "sense and 
respond" (Bradley and Nolan, 1998). New, inter-active 
communication networks enable organizations to sense, 
in real time, custo mer’s nee ds and to re spond sw iftly and 
effectively. The speed with which advances are 
currently being made in the business applications of 
information and communication technologies triggers an 
unprecedented change in organizations, industries, 
business and society such that those who study these 
changes are only beginning to understand (Bradley and 
Nolan, 1998). 

Propelled by advances in information technologies and 
socio-economic changes, many organizations have 
adopted, or are beginning to experiment with, "virtual" 
ways of working. Included in these ways of working are 
modes such as tele-commuting, working from satellite 
centres, mobile work and the like. In some 
organizations, it is no longer important when or where 
employees work, as long as the work is performed on 
time and efficiently. Theemerging organizational forms 
represent a shift in the organization of work done by 
large corporations that have created mass-production 
systems, which, in turn, have required the congregation 
of organizational employees at central places of work 
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(Garud and Rappa, 1994). The advent of information 
technologies has enabled a de-centralization of work by 
facilitating organizationmembers to work together while 
being spatially and temporarily de-coupled from one 
another. 

Although IT facilitates critical organizational functions, 
it also poses a particular challenge in virtual contexts 
(Korac-Kakabadse, Korac-Kakabadse and Kouzmin, 
1999). These are co-ordination and control of dispersed 
organizational actors; work group functioning; 
encouragement of extra-role citizenship behaviours; and 
retention of valuable employees. Achieving co- 
ordination and control in virtual organizations is 
increasingly difficultas more organ ization members are 
expected to perform functions that are not fully 
predictable, not easily measurable and require high 
levels of inter-action with others; factors which may 
complicate a worker's ability to perform virtually 
(DeSanctis, 1984). Co-ordination and control, which are 
essential ingredients to organizational efficiency and 
effectiveness, become increasingly difficult in virtual 
environments as organization member's productivity 
depends upon the ability to formulate reliable 
expectations about other's behaviour and to rely on 
others to perform the functions that they are assigned in 
a consistent and timely manner (DeSanctis, 1984). 

SYMBOLIC ANALYSTS OR 
"TECHNICIZATION” OF WORK 

Although IT is held as a liberator of knowledge, 
separated from the certainty of history, technology seems 
to re-inforce a "crisis of knowledge", in turn, often 
producinga dis-connectedness from rationality (Connell, 
1 997). In this shiftingterritory of fading knowledge, the 
sensation that there is causation seems strangely 
vacuous. Information, in its current usage, is 
utilitarianism in extremis, circumventing modernist 
proposals about where things come from (Breen, 1997). 
This fading of knowledge isaligned with the absence of 
a structured critique, producing deep confusion in 
surface discourses about information (Roszak, 1994). 
As information moves rapidly across the surfece of 
everyday lives, its dis-articulation from knowledge may 
produce a knowledge crisis. Whilst, at present, it is 
difficult to ascertain the size of the symbolic analyst 
occupation as a category, because it is often associated 
with technicians, ones who inter-act with IT or 
administrative support groups, it can be inferred that 
numbers are growing. Since mid-century, the per- 
centage of Americans employed in "service" and 
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professional occupations ranked second (89 per cent) 
and third (82 per cent) for growth, whilst growth of the 
technic ian's category has grown by 240 per cent (Barley, 
1996). Computer technicians and automobile 
technicians are grouped beneath the ’precision 
production, craft and repair occupation’ whilst computer 
operators are counted with 'administrative support 
occupations' (Barley, 1996: 410). 

One of the biggest impediments to E-business is the fact 
that computers and Webs are not very effective in 
recognizing context; thus, many Web searches result in 
unwanted information. The reason is that the Web's 
main language, HTM L (hypertext m arkup language ), is 
essentially superficial. It tells a Web browser how to lay 
out the contents of a Web page, but it remains ignorant 
of the content. The World Wide Web Consortium 
(W3C) developed an extension to HTML, called XML 
(extensible Makeup Language), which, in addition to 
describing content, also provides away of indexing data 
{The Economist, 1999a). It is a system of tagging data 
with relevant information, allowing applicationsrunning 
on the computers to respond in an appropriate way. For 
example, XML makes it clear that "The Times" is a 
newspaper and in a particular time zone. By using meta- 
tagging data that describes data, XML can also speed 
searches in the way a librarian’s card index can. 
However, in order for XML to work effectively, there is 
also a need for some agreement on definitions. This can 
be achieved within particular professions, such as 
medicine, although there is a need for a shared language 
of business, on the Internet, across industry. Microsoft 
is using its market clout to enforce global standards via 



its product, "BizTalk", which will be incorporated into 
Office, BackOffice and Windows {The Economist, 
1999a). 

THE RISE OF THE INFO-MEDIARY 

IT has had significant effects on organizations. 
Organizations are changing the way they operate due to 
significantreductions in thecost of obtaining, processing 
and transmitting information (Porter and Millar, 1985; 
Malone, Yates and Benjamin, 1987; Rockart and Scott 
Morton, 1993). Information accessibility makes radical 
changes possible in management praxis which, in turn, 
affects organizational and market structures. Although 
electronic data processing and telecommunications 
infrastructures have supported the development of 
extended global enterprises(Harasim, 1 993; Konsynski 
and Karimi, 1 993), they have, at the same time, induced 
de-integration of industrial structures, thus, resulting in 
"virtual corporations" or "networked organizations" 
(Rheingold, 1 993; Rockart & ScottMorton, 1993). This, 
in turn, facilitates continuous evolution of a firm's value 
chain of production of agood or service and distribution 
(Porter, 1985). The potential of IT to transform value 
systems of many organizations is, thus, far greater now 
than it has been in the past, as technology enables 
producers of goods and service s to directly inter-actwith 
consumers. Some producers are taking advantage of 
direct electronic links with consumers. However, short 
E-commerce history shows that this direct link is 
increasingly mediated by a new breed of inter- 
mediaries the electronic market has not eliminated 
intermediaries from the value system. 



TABLE 1 

REDUCING COST THROUGH INFORMATION EXCHANGE 



Activity 


Potential Impact of Web-Based Technology 


Administration 


Electronic transmission of orders and invoices directly between customer and suppliers B2B E-commerce. 


Inventory 


Sharing information on stock and demand to avoid an enterprise carrying unnecessary stock Just-in-Time stock 
supply. / 


Transport/Storage 


Optimizing delivery to nearest transport or storage space is utilized effectively to meet agreed service levels. . 


Design 


Sharing product design data inter-actively to enable faster developmoit of better products and less ”re-work,’’ 


Financing 


Electronic payment to improve cash flow, reduce the need for working capital and reduce costs. 


Capacity 


Matching the use of resources across firms to avoid idle resources in one part of the supply chain and/or overload 
in another. 


Services 


Linking third-party service suppliers to search requests to reduce delays in delivery and cost of administration. 
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BEST COPY AVAILABLE 



Although IT allows producers to internalize activities 
thathave been traditionally performed by inter-mediaries 
and, at the same time, "capture value" and re-distribute 
profits along the value system, inter-mediaries have not 
disappeared they have been transformed. The 
netwo rk’s ability to supp ort direct exc hanges e fficiently 
will increase both producer and consumer welfare 
(Benjamin and Wigand, 1995). Notwithstanding that it 
has been shownthatinter-me diaries add significant costs 
to the value chain, reflected in higher final prices of 
products or services, whenever a transaction takes place 
directly between producer and consumers or through 
third-party market facilitators, both producers and 
consumers benefit. The producer gets to retain ahigher 
portion of surplus value generated along the value 
system, while the consumer benefits from both a larger 
choice and lower prices for services (Benjamin and 
Wigand, 1995). 

According to Transaction Cost Theory (Coase, 1937; 
Williamson, 1975), aproducerofgoods and services has 
two options for organizing economic activities; an 
internal hierarchical structure which integrates the 
activity into its management stmcture or a market-like 
relationship with external firms (Williamson, 1975). 
When the market mechanism is at work, the flow of 
materials and services takes the form of external 
transactions and is co-ordinated by market forces. In the 
context of E-distribution channels, organizations can 
decide whether to "make or buy" (Rangan, Menezes and 
Maier, 1 992) an inter-mediary function; that is, whether 
to internalize the channel activity or sub-activitieswithin 
its organizational boundaries or choose to rely on the 
market. 

Malone et al. (1987) were among the first to link the 
Transaction Cost Theory to electronic communication, 
illustrating how electronic networks can lower the co sts 
of transactions and influence the formation of both 
electronic markets and electronic hierarchies. More 
efficienttransactions help organizations reduce the costs 
of co-ordination, defined as the transaction costs of all 
the information processing necessary to co-ordinate the 
work of people and machines that perform the primary 
processes [and] take into account the costs of gathering 
information, negotiating contracts andprotectingagainst 
the risks of "opportunistic bargaining" (Malone et al., 
1987: 485). Following this line of thought, it is held that 
networks encourage vertical de-integration of firms by 
lowering the cost of’buying" compared to "making" in- 
house (Malone et al., 1987; Malone, Yates and 
Benjamin, 1989). For this very reason, producers 
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outsource inter-mediary functions, resulting in a greater 
relianceon inter-mediaries, as the reduced co-ordination 
costs imply an "un-bundling" of functions, making it 
easier and more efficient to buy value-chain functions 
rather than to make them in-house. 

Thus, Internet, Web and related E-business technologies 
were expected to eliminate, or at least marginalize, 
dealers, distributors and other commercial go-between 
middle-men, by putting buyers and sellers of goods and 
services in direct contact, by-passing inter-mediariesand 
creating "dis-inter-mediation" of the market (Preston, 
1999). The reality is that these technologies have 
attracted hundreds of new companies whose main 
purpose is to hook up sellers and buyers, thus, a new 
kind of middle-man creatingthe "re-inter-mediation" of 
market by a new breed of middle-man the 
infomediaries (Hagel and Raypoert, 1997; The 
Economist, 1999a; 1999b). "Info-Mediaries", together 
with "aggregators", "hubs", "auction sites", "portals" and 
"exchanges", are labels used forthe E-based companies 
which aim to improve market-clearing dynamics by 
leveragingthe accessibility, efficiency and information- 
richness of the Web, each having a specific focus. 

For example, aggregators, exemplified by Chemdex, 
help buyers in fragmented markets select products by 
providing up-to-the-minute price and product 
information and a single contact point for service 
(providing a one-stop -shop for academ ic research and a 
service for companies in the pharmaceutical and bio- 
technology business to purchase all their supplies). On- 
line auctioneers, exemplified by Adaction.com, offers a 
reliable channel for sellers to dispose of perishable or 
surplus goods or services at the best possible prices and 
for buyers to get bargain prices without taking a leap 
into the unknown. It offers media buyers the ability to 
bid for specific product category via a tune-in, first- 
come, first-serve basis through "Opportunity Exchange" 
and by automating bidding through an agent called 
"ProxyMan". Exchanges, exemplified by NTE, create 
liquidity in otherwise fragmented m arkets, lower average 
stock levels by matching bid/ask offers and act as neutral 
third parties enforcing market rules and settlement terms 
{The Economist, 1999a; 1999b). 

The appeal of the modern-day middle-man, info- 
mediaries, are based on theirunique ability to exploitthe 
Internet’s three most salient characteristics. First, it 
shifts power from sellers to buyers by reducing the cost 
of switch suppliers and freely distributing a larger 
amount of price and product information (The 
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Economist, 1999a; 1999b). As a result, many buyers feel 
liberated and free to make choices; others feel 
overwhelmed by the amount of available information 
and choices. These buyers prefer one-stop-shops and are 
prepared to pay for information they know is accurate 
and trustworthy and which provides opportunities for 
third-party sellers {The Economist, 1999a; 1999b). 
Information is particularly powerful as it applies to 
business-to-business sales, mainly because business buys 
more on-line than the consumer. In the US alone, 
business-to-business E-commerce is expected to be 
US$1 .3 trillion by 2003, compared with USS 108 billion 
with consumers (Forester Research, 1999). These info- 
mediaries create liquidity by making real-time 
information available andthe price easily accessible. In 
the process, they adjust prices more closely to demand 
and they often handle settlements (Preston, 1999). For 
example. Instill Market Intelligence, which offers 
subscriptions to the purchase data generated from the 
transaction-level purchase data gathered from the 
operator, invoices on its purchase procurement 
service an E-marketplace for the food service industry. 
Access to data lets manufactures effectively manage 
production, plan new products and markets and sell 
existing products to the best-suited buyers prepared to 
pay US$100,000 per year, per product category (Hagel 
and Raypoert, 1997). 

Secondly, the Internet reduces transaction costs and, 
thus, creates economic activity. A bank transaction via 
the Internet costs one cent, compared with 27 cents at an 
ATM or 52 cents over the telephone. Processing an 
airline ticket on the Internet costs US$1, whilst, through 
a travel agent, it incurs US$8 in charges. However, 
these savings are available to large businesses, such as 
financial institutions, creating opportunities for info- 
mediaries by linking buyers and sellers via the Internet 
and achieving savings for both in markets where they 
might otherwise miss out {The Economist, 1999a; 
1999b). 

The third fact is that the speed, range and accessibility of 
information from the Internet, and the low cost of 
distribution and capturing it, create new commercial 
possibilities. Info-Mediaries, sitting in the middle 
between buyers and sellers, are uniquely placed to 
collect information, add value to it and distribute it to 
those who will find it more useful (The Economist, 
1999a; 1999b). These info-mediaries can create a 
monopoly as they use information to attract more buyers 
and sellers and, simultaneously, /earij more in the course 
of their business transactions. However, info-mediaries 



can also use information to solve particularproblems for 
a particular vertical market. By defining distinct focus, 
info-mediaries can provide a depth of information and 
drive transactions by generating customer loyalty and 
participation. Increased customer participation, on the 
other hand, increases the depth of the info-mediary’s 
knowledge base which, in turn, attracts more 
transactions. The key to modern-day middle-men is to 
add value (Preston, 1999). Web-based inter-mediation 
has a strong client base in utility services such as 
electricity, natural gas and international telecom, whilst 
it is increasingly gaining market share in the metal, 
timber, steel, transport, plastic, food service and 
computing industries (Preston, 1999). 

However, info-mediaries have access to the consumer's 
credit status as they match the individual up with the 
right service providers (Hill, 1998). The down-side is 
that info-mediaries sell consumer information to third 
parties, impinging on consumer privacy. Although there 
are honest on-line brokers who behave with integrity, 
and consumers do trust them with their personal profiles, 
the majority of info-mediaries sell consumer's data to 
others. Some have suggested that info-mediaries can 
perform a dual role ofbroker and gate-keeper, where the 
consumer will be in charge of a brand called "me" and 
would charge the info-mediary for selling this data. 
Although someconsumers would welcome such a move, 
research data suggest that 84 per cent of 10,000 Web 
users are content for providers of services and/or info- 
mediaries to sell information to other companies (Each, 
1999). Some are developing software tools that will 
capture and negotiate a consumer's commercial identity 
so that the balance of power in the sale chain can be re- 
ordered (Gerland, 1997). When a company desires to 
know a consumer's tastes or to examine past transaction 
histories, the software info-mediary will determine the 
validity of the request and will then determine somekind 
of payment for the use of the data. In the end, electron ic 
marketplaces will determine the pricing of such data. 
However, the decision as to who will accurately build 
and maintain such software agents is still debatable 
(Gerland, 1997). 

THE IMPACT OF WEB-BASED 
TECHNOLOGY ON INTER-MEDIARIES 

Web-based technology presents enterprises with choices 
as to how to positionthemselves in theanticipatedfuture 
market. Although there are many options, they can be 
clusteredinto five categories, depending on the function 
web technology is employed to perform; namely, 
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substituted/re-configured, dis-inter-mediation, re-inter- 
mediation, partial inter-mediation and complimentary 
inter-mediation. 

Substituted/Re-Conjiguredlnter-Mediaries W eb-based 
technologies can offer substituted and re-configured 
products and, thus, change or re-configure traditional 
inter-mediates. For example, an electronic channel may 
enable the underlying customer need to be satisfied in a 
differentway or by ’’substitute products", exemplifiedby 
E-mail and physical post. This substitute provides a 
threat to traditional inter-mediaries, such as paper 
manufactures, stationers and post offices. In addition, 
customer needs may be bundled into different product 
configurations or "re-configured products", such as 
newspapers, which provide news, job vacancies and 
advertisements that is, provide the bonded value of 
newspaper services with other types o f value. However, 
some portals, such as Yahoo, meet some of those needs 
and poses a threat to the traditional news inter- 
mediaries the newspapers. 

Dis-Inter-Mediation With other IT-enabled channels, 
such as call centres, Web-based technology can enable 
a link to be removed from the market by removing inter- 
mediaries whose primary function of information 
transfer can be more effectively performed using the 
Internet. Examples are telephone and Internet banking. 



direct purchasing, such as offered by Amazon.com, by- 
passing sale agents and distributors used by consumer 
goods manufactures selling direct to retailers. 

Re-Inter-Mediation Increasingly, instead of by-passing 
traditionalinter-mediaries, they are replacedby new on- 
line info-mediaries. For example, on-line sites which 
automatically search for the cheapest car insurance are 
competing with telephone-based brokers, causing the 
demise of the Automobile Association’s (AA’s) high- 
street shops in the UK. Similarly, General Electric’s 
TPN Register provides an on-line marketplace between 
suppliers, on the one hand, and GE and its partners on 
the other. There are new integrating info-mediators that 
provide integrated services, whilst others, exemplified by 
Japan Airlines, puts out open invitations for tender to 
suppliers on their web site. There are also market 
makers, such as action sites which are tied neither to the 
buyer nor the seller. Some inter-mediaries are closer to 
vendors, such as Auto-by-Tel, which passes on leads to 
local dealers. A more buyer-oriented inter-mediary, 
such as the TPN Register, sets up a consortium o f buyers 
in order to ensure that they gain low prices through 
economy of scale. There are a variety of inter-mediaries 
and, according to their orientations, may be classified as 
vendor-oriented or buyer-o riented, with market m akers 
being in neutralpositions. Web technology-based info- 
mediators are summarized below: 



TABLE 2 

TYPES OF INFO-MEDIATORS 



Intcr-Mediary Type 


Inter-Mediary Form 


Vendor-controlled 


Vendor Web site (Coachyou, E-Strategy Consulting Practice) 


Vendor-oriented 


Vendor run community (Yahoo!, Cambridge Information Net, AOL) 

• Consortium distribution (Theatreline, Euro-tec) 

• Vendor's agent (Chemdex) 

• Lead generator (Auto-By-Tel) 

• Audience broker (Double-Click, E^^Trade) 


Un-biased 


User-created autonomous multi-agent (Kasbah) 

• Market maker (eBay, Priceline, Petrocoms) 

• Shop (ShopSmartcom) 

• Mall (msn.com, bcity.com) 


Buyer-oriented 


Purchaser's agent (Comparene, Autoland) 

• Purchasing aggregator (TPN Register, InsWeb, Lending Tree) 


Buyer-controlled 


Buyer Web Site (Japan A irlines. Finder) 
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Partial Inter-Mediation Partial inter-mediation is a 
halfway house. In some cases, an inter-media ry's role 
may be reduced, but noteliminated, through some value 
being provided remotely by the supplier to the inter- 
mediary's customer. Web sites, exemplified by those of 
car manufacturers, may build a brand and provide 
customer information while pointing customers to 
traditional outlets for actual purchases. 

Complimentary Informediation Although, at times, the 
links in the value chain may remain the same, 
communication between them may be partially or fully 
switched to Web-based communication from the 
previous media or mechanism (that is, media switching 
or addition to existing communication). For example, 
Dell's addition of the Internet to its other means of 
communication with customers and RS Components, 
who have, similarly, added a Web channel to its 
dominant telephone sales model whilst selling to the 
same customer base. 

TOWARD "SEAMLESS" 
ELECTRONIC MARKETPLACES 

At the beginning of the last century, crude oil was 
nicknamed "black gold" for its capacity to forge new 
wealth. At the beginning of the 2F‘ century, the new 
gold is consumer information the data stream, data 
capture and data m ining that, with the proliferation of the 
Internet, adds to a new currency of "infonomics". 
Customer information and a variety of data analysis 
techniques save marketing overheads by eliminating 
inappropriate customers and increase "cross sell" and 
"up-seir to customers who are ready to experience new 
services or products {The Economist, 1999a). The 
marketing mantra of the last decade of the 20‘^ century 
was "segmentation", which turned into the elusive idea 
of "one-to-onemarketing", only to become a reality with 
the proliferationofinternet and Web-based technologies 
creating "mass personalization" of products and services. 
Whether on-line or off-line, enterprises pursue Pareto 's 
Law, whereby they can make 8 0 per cent of their 
revenues from 20 per cent of their existent customer 
base hence, the reason why airlines, banks and 
stockbrokers service the top one per cent of their clients 
with "deluxe" attention while the "proles" line up in 
queues {The Economist, 2000). However Pareto's Law 
assumes that the enterprise already has customer 
information. Marketing departments are increasingly 
under pressure to increase their enterprise's profitability 
by 80 per cent by improving their customer re lationship 
by 5 per cent which, in some cases, could mean a 



difference of £lbillion {The Economist, 2000). 
Capturing customers' information allows personalized 
marketing to become more important on the valuechain. 

In addition to purchasing customers information from 
third parties, Internet-based enterprises are able to 
practice explicit and implicit data collection and 
marketing. In explicit data collection, customers are 
asked questions during the initial registration process 
which they voluntarily answer (American Express, Fly 
Buys). Implicit data collection is more subtle and, 
perhaps, less ethical. The Internet site tracks what an 
individual customer or visitor is doing on thatsite; they 
follow individuals and make assumptions about what the 
individual desires (Wellman and Wurman, 1998). These 
sites leave an identifier, a "cookie", in a visitor’s 
computer, so whenever an individual returns to the site, 
the "cookie" triggers a recognition that pulls and builds 
an individual’s history. Rule-based expert system 
engines, orpersonalization engines, then decide howthe 
site should respond to an individual's browsing session 
{The Economist, 1999a). A book retailer, such as 
Amazon.com, has personalization engines that decide 
what books one may be interested in, based on previous 
titles one has bought. MyYahoo! for example, uses the 
dynam ic personalization engin e that resides in the centre 
of the Internet’s most-used directory. The information 
that MyYahoo! gathers, non-stop, as soon as a visitor 
enters one of its sites is usedto bring the user further and 
further into a community. All the features of Yahoo ! are 
personalized, from clubs and football tipping, chats and 
auctions, to the mail and message services. All features 
contain customized elements that consumers control 
(Wellman and Wurman, 1998). Hence, Yahoo is less 
likely to serve banner ads or promotions that are 
unwanted. The power of using customer inform ation in 
the manner that is appealing to each individual is the 
difference between "spam" and the "helpful offer" or 
good customer relationships {The Economist, 2000). 

In addition to Internet-based data collection, many 
enterprises collect customer information via "call 
centres" as well. Calling Line Identifier (CLI) software 
sends a caller's number to a customer database andlooks 
for a match with its records. Th e search is au tomatica lly 
narrowed down to segments as the Web site shows 
different call numbers depending on who browses the 
site. 

The power of personalization and the ability of the 
Internet have opened many possibilities for marketing. 
Increasingly, "old" Web developers are becoming 
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business integrators who build complex, data-based E- 
commerce sites founded on an information architecture 
that utilizes the Internet from the perspective of the type 
of information the enterprise wants and the type of 
presentation likely to elicit it from customers. 
Increasingly, many portals are being replaced by info- 
mediary sites. Portals are generally understood to mean 
a starting poin t through w hich one e nters the W eb but, in 
the business-to-business (B2B) arena, it usually means 
entrance to a Net marketplace where users are provided 
with news and value-added services related to the 
business or industry th e marke tplace serve s. Electronic 
marketplaces are Web sites where users go to buy and 
sell. Marketplaces are formed by collections of software 
agents which inter-act with one another in order to 
automatically negotiate and form partnerships or trade 
products and services through the Internet (Nomura 
International Pic, 2000). Whilst the marketplace's main 
role is to facilitate inter-action between agents, it may 
play other roles depending on the type of market. For 
example, a marketplace serving a particular industry 
might display commodity prices and may contain 
discussion groups or bulletin boards related to the 
interests of purchasers that it would like to attract. It 
also offers a searchable database of the products 
available from the suppliers participating in the 
marketplace (Wellman and Wurman, 1998). It also 
determines the terminology spoken; that is, how goods 
and services are described. As original portals were not 
designed to collect customer information they are now 
being upgraded or replaced by info-mediary engines. 

Hence, the "owners" of E -space in the next few years 
will be huge database enterprises or electronic info- 
mediaries that act on behalf of customers to source the 
most effective goodsor services. Through the spending 
power they represent, these sites will be able to gain 
'discounts and special offers for users. For example, 
InsWeb insurance polices and LendingTree for home 
loans, will find the best deal available on the Internet. 

CONCLUSION 

The idea behind the electronic marketplace is that the 
product content and the necessary infrastructure are 
provided by someone else and not by the market site. 
Venture capitalist, software providers and, most 
significantly, purchasing organizations themselves, 
usually in industry-wide collaborations that include 
players from every phase of the supply chains, are 
embracing net-marketplaces (Jennings, Sycura and 
Wooldridge, 1998). There are currently more than 600 
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Net marketplaces in existence and it is expected that by 
2003 there will be as many as 10,000. 



TABLE 3 

ELECTRONIC MARKET-PLACE 
DESIGN CHALLENGES 



Challenge 

Type 


Exception Class 


Unreliable 

infrastructures 


Infrastructure failure (slow host, 
agent death, lost/garbled 
messages). 


Limited trust 


Protocol violations (fraudulent 
bids, agent skills change, 
contractor refuses payment). 


Systemic 

failure 


Systemic expressions (no capable 
agents, biased bid selections, no 
available agent due to resource 
poaching by lower priority tasks). 



Source: Chia, Neiman and Lesser(1998) 



Whilst Internet, Web and related E-business 
technologies where expected to eliminate, or at least 
marginalize, dealers, distributors and other commercial 
go-between middle-men, by putting buyers and sellers of 
goods and services in direct contact, by-passing inter- 
mediaries and creating "dis-inter-mediation" of the 
market, the current re ality is that these technologies have 
attracted hundreds of new companies whose main 
purpose is to link sellers and buyers, creating new kinds 
of middle-men and creating "re-inter-mediation" of 
markets by a new breed of middle-men the info- 
mediaries (Wellman and Wurman, 1998). The appeal of 
modern-day middle-man info-mediaries is based on their 
unique ability to exploit the Internet's three most salient 
characteristics;namely,the potential to shiftpower from 
sellers to buyers, potential reduction o f transaction costs 
and speed of transactions. An example is the banking 
sector where distribution of banking products is being 
dis-inter-mediated with the support of 
technology starting with automated teller machines 
(ATMs), telephone banking, PC-based proprietary 
systems and the Internet, only to be replaced by info- 
mediatory and personalized services. Further info- 
mediation will be experienced in the banking sector 
when Internet and related technologies by-pass banks in 
payments, clearing and settlement (Preston, 1999). 
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However, the design of efficient and robust electronic 
marketplaces, whose participants are independently 
developed software agents, is a challenging task the 
main challenges being unreliable infrastructure, limited 
trust and systemic failures (Chia, Neiman and Lesser, 
1998). The organizational design implications of these 
vulnerabilities leave a great deal of uncertainty for inter- 
organizational(Net) transactions, as theydo forsoftware 
development seekingto further re-engineertransactional 
dependancies. 
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A focused exploration of research presented atlAlM in 1997, this study applies the structuring of student teams 
recommended by Mennecke and Bradley (1997), adapts the roles, and extends structure to group meetings. Findings 
verify the increased team cohesion found by these prior researchers and, in addition, indicate that the use of meeting 
constraints as well leads to a more satisfying team experience. Recommendations are providedforstructuringproject 
teams so that procrastination is minimized and equal contribution to the effort is facilitated. 



INTRODUCTION 

The goal of this study is to extend the work of Mennecke 
and Bradley (1997) concerning the impact ofstructured 
roles on student team projects. Mennecke and Bradley 
(1997) address the problems associated with working in 
project teams for IS student projects: 

• Students who choose to engage in group social 
activities prefer to work alone on projects. 

• Teams require higher transaction costs in terms of 
time to complete the project, communication, and 
scheduling. 

• Teams "procrastinate until the project due date" and 
"as the pressure increases, group cohesiveness 
breaks up" (p. 23). 

Working with traditional students who lack work 
experience, they find students unskilled in team process. 
Therefore, they test structuring roles for project teams, 
and measuring results after controlling for many 
confounding variables. While their instruments do not 



find measures of satisfaction to be higher in the 
treatment group, they find that structuring roles 
improved the quality ofthe projectsand improved group 
cohesiveness. They conclude that the added structure 
improved team process. They believe thatincreased use 
of "e-mail, list serv, chat room" (p. 24) would further 
improve team process and lower transaction costs. 

Mennecke andBradley (1997) provide some structure by 
requiring that each student assume one key role within 
the team. Their findings and presentation of these 
findings stimulated the current adaptation of the roles. 
In the current study, students are required to vary 
assigned roles over the course of the project. 
Preconditiondata is gathered, e.g., participation on prior 
project teams, work experience, preference for working 
alone or in teams, perceptions regarding the 
effectiveness of working in teams and transaction costs 
of team work. Not only are roles structured, but the 
conduct and scheduling of meetings is structured. 
Anecdotal evidence suggests that failure to meet 
throughout the semester and failure toconduct meetings 
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effectively adversely affects team process. Meetings are 
a major source of team conflict. Further, documentation 
of team process are made a component of project 
deliverables and evaluation for the treatment group. 
After projects are complete but before projects are 
graded, students complete a questionnaire concerning the 
team experience as well as open-ended questions 
concerning the team process. The control or 
comparison group is another database class in which 
these team process constraints are not employed but 
which uses the same syllabus for the course and the same 
requirements for the database project. 

Our objective is to recommend methods for improving 
student team process, beyond the increased team 
cohesion demon strated by Mennecke andBradley (1997) 
through the use of structured roles. We want these 
methods to lead to some degree of satisfaction with the 
team process. Further, we want to use the team project 
to teach effective meeting skills. 

METHODOLOGY 

Students enrolled in two sections of a required course in 
Database Systems participated in the study. One section 
appliedthe treatment described above; theother required 
team projects but did not structure the team interactions 
or even describe the possibility of role assignment or use 
of meeting agendas, action lists, and minutes. At the 
beginning of the semester, the treatment class is given a 
description of team process constraints, involving both 
meetings and structured roles. As recommended by 
Mennecke and Bradley (1997) in their future research 
section, extensive use of e-mail is required to facilitate 
communication and coordination. 

• Project teams are to meet at least once per week, 
beginning with week 2 of the semester. A regular 
meeting time at the same place, time, day is strongly 
recommended. Teams may need to meet more than 
once per week during some phases of the project. 

• Each project team member must play one of the 
threeprimary roles during the course of the semester 
and each of the three roles must be designated at 
each meeting. The three roles are meeting fecilitator, 
scribe, and scheduler. 

• Each meeting must have an agenda provided by the 
meeting facilitator to each team member prior to the 



meeting. E-mail is recommended for this task. Each 
meeting must result in an action list designating 
tasks to be performed, responsibility for the task, 
and deadline for the task. The first item on every 
meeting’s agenda should be a review of the action 
list from the prior meeting. Each meeting’s agenda 
should also include formal recognition of the 
approval of the minutes of the prior meeting. These 
minutes can be approved via e-mail after all 
necessary revisions aremade. The scheduler should 
always be on each agenda, reporting on project 
status against the timetable. 

• The scribe must record the minutes of each meeting 
and these minutes are an essential part of the project 
deliverables. The minutes must include the 
following items: m embers present, m embers absent, 
members playing each of the three primary roles, 
copy of the agenda, detailed notes on 
discussions/decisions for each agenda item, the 
action list for the next meeting. The action list for 
the next meeting must be distributed to each team 
member via e-mail within two days of the meeting. 
Minutes must also be distributed to team members 
for approval via e-mail no more than two days after 
each meeting. 

• The scheduler must contact each team member at 
least two days prior to each meeting to confirm 
meeting attendance, reschedule meetings when 
attendance is problematic, and maintain the project 
timetable. The scheduler should report the status of 
the project against the timetable at each meeting. 
The team may wish for each member to serve as 
scheduler for two consecutive weeks. 

After this descriptio n of team process is presented, each 
member of the class is asked to briefly introduce himself 
and to indicate good meeting times. Students chose their 
own team members after these introductions. Students 
in the control group also chose their own team members. 

These project team constraints extend those specified by 
Mennecke and Bradley (1997, 21), where four roles are 
defined. 

• Presider or Meeting Leader 

• File Manager or Project Leader 

• Meeting Coordinator 

• Intermediary 



160 



Proceedings of the 15^ Annual Conference of the International Academy for Information Managemait 



We employ three roles which are parallel but not 
precisely equivalent to their first three roles. 

• Facilitator 

• Scribe 

• Scheduler 

The facilitator like the Presider or Meeting Leader 
develops and distributes a meeting agenda and is 
responsible for keeping the meeting on task. No Project 
Leader was designated but documentation, a primary 
responsibility ofthe File Manager or Project Leader, was 
the scribe’s role for meetings. Project documentation 
would have been an action list item, assigned by 
consensus at the meeting. The Meeting Coordinator 
tasks were filled primarily by the Scheduler. The 
Scheduler also kept the project schedule or time table. 
Regularly scheduled weekly meetings were held, so no 
meeting coordinator needed to call a meeting. The 
Itermediary role, where a student was assigned to act as 
the intermediary between the project team and the 
instructor, was not employed. The roles have many 
similarities, but perhaps most importantly, students have 
well defined roles to play in both studies. 

However, in this study, students must play all roles 
whereas roles were assigned for the entire project in the 
Mennecke and Bradley (1997) study. Our rationale is 
that students need to learn all three roles and that 
students who have played a role will tend to be more 
cooperative with other students playing this role. The 
major extension to team structure is to specify weekly 
meetings with an agenda, published minutes, and action 
list. This structural component should work to prevent 
procrastination, a major problem with student team 
projects. 

RESULTS 

The following research questions are examined through 
a two-part questionnaire completed at the close of the 
term project (Appendix A). By analyzing these 



perceptions by students, we will consider the following 
research questions: 

Q1 . Is the tendency to procrastinate reduced through the 
increased structure of team processes? 

Q2.Do the students perceive the use of meeting 
constraints to be valuable? 

Q3. Do the students perceive the assignment of roles to 
be valuable? 

Q4.Do the students in the treatment group experience 
higher satisfaction with the team process? 

We will alsoconsider mitigating factors such as declared 
work experience, previous project experience within a 
class, and overall preference for team projects. Finally, 
recommendations for improving team process for IS 
student projects will be formulated, applying and 
building on the work of M ennecke and B radley (1997). 
They demonstrated that group cohesion was positively 
impacted by their treatment but that satisfaction with 
“group process, satisfaction with the group’s project, and 
group member ratings” (p. 22) were not significantly 
different in the treatment group. 

MITIGATING FACTORS 

Age and Experience 

The age, number of prior information systems courses, 
and work schedules of the two groups were ve ry similar. 
However, the control group, a slightly older group, had 
significantly more experience in information systems 
work, and of those who had such experience, the control 
group had a greater percentage of members who had 
been on development teams. This IS experience, 
coupled with developmentexperience, should predispose 
this group to have a better project experience. 
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TABLE 1 

AGE AND EXPERIENCE 



ATTRIBUTES OF AGE AND EXPERIENCE 


Control 
Mean 
Std. Dev. 


Treatment 
Mean 
Std. Dev. 


T-test 
p value 


1. 


Average Age 


29.73 


27.55 


.14 






7.71 


8.35 




2. 


No. of prior IS courses 


5.1 


4.16 


.35 






3.85 


2.87 




3. 


Work full time 


33.33% 


30% 




4. 


Work part time 


43.33% 


45% 




5. 


Do not work 


23.33% 


25% 




6. 


Worked in IS (y/n) 


43% 


16% 


.01 


7. 


If y to #6, part of a development team? (y/n) 


35% 


8% 


.009 


8. 


No. of IS projects completed 


2.2 


3.21 


.054 






1.47 


1.51 





Overall preference for team projects. Students in b oth 
the treatment and control group agreed that they like to 
work on a project team and liked to work alone. The 
treatment class actually showed a stronger preference for 
working as a member of a project team, but that 
preference was not statistically significant compared to 
the control group . Both groups agreed thatprojects take 
more time if completed as a team activity. However, 88 
percent of the students agreed with the statement that 
“Project quality is improved if the project is a team 
activity ."There were no significant differencesbetween 
the two groups in regard to preference for team projects 
versus working alone. 

PERCEPTIONS REGARDING 
MEETING CONSTRAINTS 

Weekly Meeting Time 

The treatment group was required to meet weekly 
beginning with the second week of the semester. The 
objective of this constraint was to facilitate incremental 
work and thus, reduce procrastination. In class, the 
instructor strongly recommended establishing a regular 
weekly time and place, so that s tudents co uld work the ir 
schedules around this regularly scheduled meeting. 

Students perceived this practice to be highly effective 
means of preventing procrastination. The meanresponse 



to the statement, “Establishing a weekly meeting time 
helped the team to work on the project throughout the 
semester instead of as a ‘last minute’ endeavor,” was 
4.43 with a .68 standard deviation. The mode was 5.0 or 
strongly agree. 

Agenda 

In general, treatment students agreed that“Publishing an 
agenda for each meeting was valuable,” with a mean 
response of 3.8 and standard deviation of .89. Ten 
percent of the students disagreed that the agenda was 
valuable, twenty percent were indifferent, and 70% 
agreed that the agenda was valuable. Only twenty 
percent strongly agreed, however. A more focused 
statement, “Publishing an agenda for each meeting 
caused us to use meeting time more effective,” received 
a similar respon se: me an of3.9 with a standard deviation 
of .91. 

Action List 

Treatment students tended to agree that the action list 
was valuable and that it helped the team stay organized. 
Agreement was not as strong or consistent concerning 
the action list’s impact on everyone doing his or her fair 
share. However, only one student disagreed with this 
third statement. Half the students did find the action list 
helpful for distributing work equally. 
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TABLE 2 

PERCEPTIONS REGARDING PROJECT TEAMWORK 

5= Strongly Agree, l=StrongIy Disagree 





Control 

Mean, 

Std. 

Deviation 


Treatment 

Mean, 

Std. 

Deviation 


T-test, 
p value 


I like to work on a project team. 


3.80 


3.75 


.84 




1.16 


.97 




I like to work alone. 


3.60 


3.3 


.22 




1.10 


.80 




I prefer to do project work alone. 


3.20 


2.65 


.13 




1.37 


.93 




I prefer to do project work as a member of a team. 


3.50 


3.90 


.24 




1.25 


1.02 




Projects take more time if completed as a team 


3.10 


2.95 


.81 


activity rather than as an individual activity. 


1.45 


1.32 




Project quality is improved if the project is a team 


3.10 


3.5 


.36 


activity. 


1.47 


1.28 





TABLE 3 

PERCEPTIONS REGARDING ACTION LISTS 



Statement 


Treatment 
Mean 
Std. Dev. 


Creating an action list was valuable 


4.3 




.57 


Creating an action list helped us to stay organized. 


4.4 




.68 


Creating an action list helped us to make sure everyone did his or 


3.85 


her "fair share ." 


.93 



Minutes 

Not surprisingly, minutes were not altogether popular. 
On the scale of 1-5 with 3 being indifferent and 5 
strongly agree, treatment students agreed (mean 3.75, 
std. dev. 1 .07) on their value and agreed (mean 3.65, std. 
dev. 1.18) that recording and publishing minutes 
improved communication. Notably, action lists were 
perceived to be more valuable than minutes. 



ROLES ASSIGNMENT 

Three statements were designed to assess whether the 
student perceived playing each of the three roles to be of 
value. One of the differences with the Mennecke and 
Bradley (1997) study was that students had to play all 
roles. Then, the stu dent was asked to assess whether the 
use of roles improved team process. Finally, students 
were asked to assess the value of each role . In general, 
treatment students agreed that playing a role, particularly 
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that of facilitator, had beena valuable experience. They 
perceived that the use of the three roles improved team 
process, with the role of facilitator being perceived as 
improving team process the most. Interestingly, only one 
student disagreed with each of the seven statements 
concerning the value of roles. 

Perceptions of Im pact of Team Process Constraints 

Both the control and treatment groups perceived that 
team skills were improved by the project and that they 
had learned to use meeting time more effective. 
Stronger agreement occurred in the treatment group, but 
differences are not statistically significant (p= .41, .36). 
The treatment group agreed with the statement that the 
constraints had increased team effectiveness. 



Satisfaction with the Project Experience 

Mennecke and Bradley (1 997) demonstrated that the use 
of roles increased group cohesiveness. The authors have 
observed that when students are unhappy with the 
project results orthe experience itself, “finger pointing” 
occurs and students are unhappy with their group. 
Therefore, as an assessment of satisfaction, we asked 
students to agree or disagree with the statement, “I 
would choose the sameteam members again.” Treatment 
group students agreed with the statement and were very 
close to strong agreement Even though the control 
group should have been predisposed to work well in 
teams because of their higher percentage of students w ith 
work experience and IS development experience, the 
treatment group actually ended the project more willing 
to work with their team again. 



TABLE 4 

PERCEPTIONS REGARDING ROLES ASSIGNMENT 



Statement 


Mean, Std. Dev. 


Playing the role of facilitator was valuable to me. 


3.65, .81 


Playing the-role of scribe was valuable to me. 


3.55, .83 


Playing the role of scheduler was valuable to me. 


3.40, .75 


Having a designated scheduler, scribe, and facilitator improved the 
team process. 


4.10, .72 


Having a designated scheduler improved the team process. 


3.65, .75 


Having a designated scribe improved the team process. 


3.85, .75 


Having a designated facilitator improved the team process. 


3.95, .76 



TABLE 5 

IMPACT OF TEAM PROCESS CONSTRAINTS 



Statement 


Control 


Treatment 




Mean, 


Mean, 




Std. Dev. 


Std. Dev. 


My skill at working on teams has improved as a result of this 


3.83 


4.05 


experience. 


.93 


.89 


I learned how to use meeting time more effectively. 


3.83 


4.05 




.95 


.69 


The project team worked more effectively with the constraints 


N/A 


3.90 


concerning meetings and roles played than otherteams on which 




.72 


I have worked which had no team process constraints. 










164 



Proceedings of the 15*^ Annual Conference of the International Academy for Information Management 




174 



TABLE 6 

SATISFACTION WITH PROJECT EXPERIENCE 



Control 


Treatment 


T-test p value 


Mean, Std. Dev. 


Mean, Std. Dev. 




3.24 


4.35 


.003 


1.41 


.93 





SUMMARY OF RESULTS 

We posed four research questions: 

Q 1 . Is the tendency to procrastinate reduced through the 
increased structure of team processes? 

Q2.Do the students perceive the use of meeting 
constraints to be valuable? 

Q3. Do the students perceive the assignment of roles to 
be valuable? 

Q4.Do the students in the treatment group experience 
higher satisfaction with the team process? 

In answer to these questions, we found regular, weekly 
meetings tended to prevent waiting until the last minute. 
Students did perceive the use of all other meeting 
constraints to be valuable, and found the action listto be 
most valuable. Likewise, students perceived the use of 
roles to be valuable to themselves and to the project 
team, particularly the role of facilitator. Students in the 
treatment group would work with the same team again, 
satisfaction with the team process. Satisfaction with the 
team process was the strongest difference experienced in 
the control and treatment group. This finding extends 
Mennecke and Bradley’s (1997) finding that cohesion 
increased with the use of roles. 

STUDENT RECOMMENDATIONS FOR 
IMPROVING TEAM PROCESS CONSTRAINTS 

Recommendations for improving team process 
constraints further include the following: 

• Student comments reveal that changing roles 
throughout the semester became burdensome. They 



recommend having everyone play each of the roles 
for at least one week, then assign the roles for longer 
durations. Many students found the constant shift in 
roles confusing. 

• Students recommend a midterm project review and 
submitting weekly team process documentation for 
feedback and possibly, grading. They would prefer 
incremental feedback. 

• Thirdly, many students recommended a team 
contractto insure agreementon meeting times, roles, 
etc. 

Some students commented that they found the 
constraints time consuming and burdensome, but the 
tenor of the comments was that students had found these 
constraints effective and used them voluntarily for 
projects in other courses. 

LEARNING CONTENT 

The objective of a project is two fold: students learn to 
work well on project teams and students reinforce course 
content by actually applying what is learned. Both 
groups were asked to assess whether or not the project 
helped them to learn content and thus affected their 
course grade. Interestingly, both groups perceived that 
the project had helped learn course content with the 
treatment group having a slightly stronger perception of 
learning content. However, neither group agreed that 
exam scores had been improved by their project work. 
This statement may confound satisfaction with the grade 
on exam scores with learning content through the 
project. 



Proceedings of the 15*’ Annual Conference of the International Academy for Information Management 



165 




175 



TABLE 7 

LEARNING CONTENT 



Statement 


Control 


Treatment 


T-test 




Mean 


Mean 


p value 




Std. Dev. 


Std. Dev. 




The project helped me to better understand 


3.97 


4.35 


.12 


topics and problems in the database course. 


1.16 


.59 




My participation on the project team improved 


3.3 


3.45 


.42 


my exam scores. 


1.12 


.89 





LIMITATIONS OF THE STUDY 
AND CONCLUSIONS 

This study is primarily a qualitative study further 
exploring and applying the work presented by Mennecke 
and Bradley (1997). The two classes involved in this 
study are insufficient for reaching broad, general 
conclusions. While assumptions of randomness are not 
met, we have used statistical comparisons of groups as 
some indication of whether differences are significant. 
Because of the difficulty of comparing project results 
when two instructors assign the grade and when team 
process was a major grade component in the treatment 
group but not the control group, no objective measure of 
project quality has been used at this time. 

The use of meeting constraints, particularly regularly 
scheduled weekly meetings and action lists, were 
perceived as valuable as was the use of roles. Regularly 
scheduled meetings did tend to reduce procrastination. 
The facilitator role was perceived as particularly 
important. Significantly, students operating under team 
process constraints were more willing to work together 
again, thus these constraints served to reduce the 
dissatisfaction often associated with student project 
teams. As an instructor, dealing with the students in the 
treatment group was much pleasanter because only two 
team members came to the instructor with team 
problems during the entire semester. As a result of this 
experiment, we recommend the use of meeting 
constraints and roles along with the following 
modifications: 

• mid-term, project review. Teams review each 
other’s work, then each team presents the project in 
class 



• weekly submission of the action list along with 
results of prior week’s action list and minutes of the 
weekly meeting 

• each student plays each role at least one, then roles 
are assigned forthe semester 

• optional contract, with a discussion of possible 
contract points. 

As Mennecke and Bradley point out, “Research of this 
type is always confounded by extraneous factors that 
cannot be eliminated” (p. 23). To some extent, what 
we have accomplished is a structured trial-and-error. 
But we have demonstrated that certain team process 
constraints do seem to make a positive difference for 
team process, student satisfaction, content learning, and 
instructor management of project teams. 
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APPENDIX A 

TEAM PROCESS EVALUATION 



Name 

Part 1: Using the following scale, pleaseevaluate the following statements. Please mark the accompanying Scan-tron 
form. Please record your name on the Scan-tron fonn as well as this form. 



1. I like to work on a project team. 

1 . I like to work alone. 

2. I prefer to do project work alone. 

3. I prefer to do project work as a member of a team. 

4. Projects take more time if completed as a team activity rather than as an individual activity. 

5. Projects take less time if completed as a team activity rather than as an individual activity. 

6. Project quality is improved if the project is a team activity. 

7. Establishing a weekly meeting time helped the team to work on the project throughout the semester instead of as a "last 
minute" endeavor. 

8. Publishing an agenda for each meeting was valuable. 

9. Publishing an agenda for each meeting caused us to use meeting time more effectively. 

10. Creating an action list was valuable. 

1 1. Creating an action list helped us to stay organized. 

12. Creating an action list helped us to make sure everyone did his or her "fair share." 

13. Recording and publishing the minutes of meetings was valuable. 

14. Recording and publishing the minutes of meetings improved communication. 

15. I learned how to use meeting time more effectively. 

16. My skill at working on teams has improved as a result of this experience. 
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17. The project helped me to better understand topics and problems in the database course. 

18. My participation on the project team improved my exam scores. 

19. Playing the role of facilitator was valuable to me. 

20. Playing the role of scribe was valuable to me. 

21. Playing the role of scheduler was valuable to me. 

22. Having a designated scheduler, scribe, and facilitator improved the team process. 

23. Having a designated scheduler improved the team process. 

24. Having a designated scribe improved the team process. 

25. Having a designated facilitator improved the team process. 

26. The project team worked more effectively with the constraints concerning meetings and roles played than other teams on 
which I have worked which had no team process constraints. 

27. I would choose the same team members again. 

Note: Questions 8-1 5, 20-27 were omitted in the comparison class questionnaire, A question regarding meeting frequency 

was added. 

Part 2: Please answer the following questions in the space provided. 

1 . Number of prior information systems courses at KSU and other schools 

2. Age 

3. Work full time? Part time? 

4. Number of other courses taken this semester: 

5. Have you ever worked in information systems? 

6. If so, have you been part ofa systems development team? 

7. If you answered yes to question 5, please describe your information systems work experience briefly. 
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The following questions apply to information systems course work only: 



1. How many information systems projects have you completed? 

2. How many projects allowed you to select your own team members? 

3. If team members were assigned, what was the basis for assignment? 

4. On how m any projects were project team s asked to follow certain guidelines to improve team process? Please describe team 
process guidelines for prior project teams. 

5. If guidelines have been given, which guidelines did you find effective? 

6. What suggestions would you make to further improve team process for the database project? 

7. If you do not prefer to work on a project team, please explain this preference. 

Other comments? 
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ABSTRACT 

In this paper, we discuss our experiences in designing and implementing a ” virtual team '' systems development 
project that enabled us to incorporate the different types of knowledge relevant to teaching Information Systems 
Development (ISD) and the different stages of learning applicab le to students in institutions of high er education. W e 
outline some of the unique advantages as well as potential pitfalls that professors interested in adopting this project 
structure should be awa re of We also provide a framework that can guide the design and evaluation of ISD courses. 



INTRODUCTION 

With the rapid developments in electronic 
communication and computer technologies, there have 
been many predictions and much hype about how these 
technologies will revolutio nize the educationalprocesses 
of universities in different disciplines (e.g., in business 
administration). However, based on a preliminary 
analysis of technology initiatives in business schools, 
Leidner and Jarvenpaa (1995) concluded that “initial 
attempts to bring infonuation technology to management 
education follow a classic story of automating rather 
than transforming... In the absence of fundamental 
changes to the teaching and learning process, such 



classrooms may do little but speed up ineffective 
processes and methods of teaching. 

In this paper, we discuss our attempt to transform, not 
just automate, the delivery/ facilitation of our classes 
using an innovative project design requiring students to 
work in “virtual systems development teams. 
Specifically, we cover the theoretical perspectives that 
have informed our thinking on teaching information 
systems development (ISD), the objectives guiding the 
project design, the tangible and intangible benefits 
students have experienced (and reported to us), potential 
pitfalls in conducting such projects, and finally, future 
directions. 
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THEORETICAL PERSPECTIVES 
Our Own Backgrounds 

Human action often reflects the theories held by the 
concerned actor regarding the domain of application 
(Argyris and Schon, 19 78). By extension, this implies 
that the theoretical assumptions and beliefs that an 
academic holds dear, informs not only how the person 
conducts research, but also how he/she teaches or how 
he/she believes a course must be taught. We (the three 
co-authors) discovered how similar our concerns and 
aspirationswere regarding theclasseswe were teaching, 
even though we were based in different universities in 
two different countries. We could trace our common set 
of concerns to our similar (though not identical) meta- 
theoretical and epistemological commitments in our 
individual research programs, which in a broad sense 
may be characterized using terms such as interpretive, 
constructivist, sociotechnical, process-oriented, non- 
deterministic and emergent, conceptual, and action- 
oriented' (Markus and Robey, 1988; Lee, 1991; 
Orlikowski and Baroudi, 1991). On self-reflection, we 
found that we enacted (and sought to further integrate) 
these beliefs in our classrooms as we taught ISD to our 
students. 

Based on our exposure to and commitment to socio- 
technical thinking (Mumford, 1995) and interpretivism 
(Orlikowski and Baroudi, 1991), we were convinced that 
systems design and implementation should not be 
viewed as a technical activity alone, but as an activity 
that is simultaneously social and technical (Davis et al., 
1992). Ratherthan consideringthe system s requirements 
as unambiguously known, we chose to view require- 
ments analysis as the most significant road -block in 
designing and implementing a successful information 
system. Having adopted this view, we could see why 
Churchman and Schainblatt (1965) saw mutual 
understanding as an essential ingredient of successful 
systemsimplem entation. Thus, we needed to incorporate 
some mechanism in our classes for future systems 
designers (i.e., our students) to develop skills of 
communicating and interpreting symbols. These skills 
would enable them in their future role as systems 
designers to attain a state of mutual understanding with 
other relevant social groups (say, the users) whose 
members did not share the designers’ frames of 
reference, and vice versa. 



Types of Knowledge 

We also realized, drawing on our epistemological 
beliefs, that the practice of information systems 
development (ISD) requires different types of 
knowledge, some of which were being consistently 
under-emphasized within the traditional models of IS 
teaching. Drawing on the work of Rockart and Scott 
Morton (1975), Lee (1991), Hirschheim and Klein 
(1989), and (most significantly) Astley and Zammuto 
(1992), we came to identify three broad forms of 
knowledge in the domain of ISD: 

(1) Facts and definitions are the presumed truths and 
objectivedescriptionsof technologies and concepts. 
Straightforward answers to the following questions, 
for example, would be included in this category. 

WhatisaDSS? 

Define generalization/specialization. 

State the advantages of a “data-oriented approach 
to systems design over a “process-oriented 
approach. 

When is a relation said to be in BCNF? 

In imparting this kind of knowledge, the role of the 
professor is to act as a provider and/or conduit of 
information. Mastery of such knowledge by students 
involves more “memorization than “understanding. 

(2) Knowledge of technology and techniques refers to 
the instrumental or “how to knowledge that allow 
students to solve certain class of problems that fit the 
structure orutilizethe easily apparent capabilities of 
the tools and techniques. In a sense, the role of the 
professor imparting this type of knowledge is to act 
as an “expert, who would illustrate the technology 
and techniques through the use of “paradigmatic 
problems which can serve as problem-solving 
templates for students. Examples of technology/ 
techniques knowledge include: 

When you wish to represent a relationship between 
a many-to-many relationshipand another entity, use 
aggregation. 

To transform a 2NF relation to 3 NF, remove the 
transitive dependencies. 

The PERT and GANTT charts can be drawn in 
Microsoft Project in the following way... 
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It is important to note that this knowledge goes beyond 
definitions, since the above statements are not very 
valuable unless the student actually learns to use the 
knowledge contained in the statements for problem 
solving. Also, it is worth mentioning here that the 
traditionaleducational model assumes that by providing 
students with “drills of solving simplified problems that 
fit the techniques/technology, they would become 
capable of using this form of knowledge in a different 
context (i.e., the “real world ). 

(3) Social knowledge: As mentioned above, owing to 
our sociotechnical orientation, we recognized the 
importance of not only the above forms of 
knowledge but also another category ofknowledge, 
which we call “social knowledge. This form of 
knowledge, in our context, refers to a practicing 
systems analyst/designer’s understanding of social 
phenomena associated with ISD, and the knowledge 
that would enable him/her to explain and predict 
behaviors of social entities (organizational units, 
different stakeholders, etc.) Social knowledge can 
further be divided into: 

3a) Propositional knowledge: This type of 

knowledge is ihstrumentalin nature andfocuses 
on the means that lead directly to some pre- 
defined ends in the social arena. Knowledge 
having the structure: a) If <specific problem>, 
then <specific soIution>; or b) outcome = 
Function (factor 1 , factor 2,... ., factor n)may be 
viewed as proposition al. This type o f knowle dge 
is most popular since it is perceived to satisfy 
the usefulness/applicability in the “real world 
criteria on superficial examination. Teaching 
propositional knowledge requires the teacher to 
play the role of an expert who articulates well- 
defined rules and then illustrates the rules 
through carefully formulated applications(real- 
world cases simplified to fit the knowledge) or 
specific success stories arising from the use of 
such knowledge. Examples of such knowledge 
include: 

Communication leading to mutual 
understanding is the key to effective 
information requirements analysis. 

A prerequisite to successful systems 
implementation is a “fit between the 
technology being implemented and the 
surrounding social system. 



3b) Conceptual knowledge: This refers to concepts 
that allow us to see the world in different ways. 
This knowledge is not directly applicable, but 
needs to be merged with a specific context and 
processed in the mind to derive implications for 
action. In other words, conceptual knowledge 
needs to be converted to instrumental 
knowledge by the knowledge users themselves 
before it can be used. It is our belief that most 
issues relating to ISD are so complex and so 
dependent on the unique context, that it is 
impossible to create universal propositions or 
techniques that are directly applicable in any 
context. In such a circumstance, conceptual 
knowledge becomes extremely valuable. 
Teaching conceptu al knowledge involves acting 
as a facilitator in the process of exploring 
abstract concepts and theories jointly with the 
students. The objective of teaching conceptual 
knowledge is not to provide ’’information” or a 
"set of rules,’’ but to help each student create 
(possibly) unique skeletal non-deterministic 
conceptual schema in his/her mind, or to assist 
in the integration of a new set of concepts in 
his/her existing schema. As students gain 
experience in the "real world," they iteratively 
modify their schema, and add more "flesh" to 
the skeletal framework. Eventually, they 
become "experienced professionals," having 
developed a very rich schema that is based on 
the conceptual knowledge fundamentally gained 
in school but enriched through reflective 
practice. 

Topics such as the Leavitt’s diamond model of 
organization, social construction, the 
hermeneutic circle, metaphor of “text to 
understand action, etc., are examples of 
conceptual social knowledge that could enable 
students to design better systems (through better 
information requirements elicitation) and to 
implement systems more effectively. 

3c) Finally, we discuss symbolic knowledge. 
This refers to two related types of 
competencies: 

The knowledge that is used to legitimize 
actions/decisions: As Information Systems 
professors, we must recognize the 
possibility that our students go to work in a 
world that is governed less by principles of 
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technicalrationality and more by power 
and politics. If such is the case, then 
access to (and application of) symbolic 
knowledge becomes very critical to the 
success of our students. In a political 
work environment, knowledge 
containedin privileged sources (such as 
Harvard Business Review, article by 
Bill Gates or James Martin, etc.) 
becomes important not because of their 
rational and conceptua 1 merits 
necessarily, but because of the 
retrospective justification that such 
sources can provide fora chosen course 
of action. 

The use of/interpretation of symbols: 
Perhaps, a different bu t more re levant aspect 
of symbolic knowledge is that to be an 
effective participant in ISD, it is important 
to develop the skills for using and 
interpreting symbols (utterances, text, 
pictures, etc.) in an effective manner (e.g., 
Sarker and Lee, 1999). 

As IS researchers interested in social analysis, our 
exposure to the tradition of symbolic interactionism 
allowed us to see the importance of this form of 
knowledge in ISD. From this perspective, therole of the 
professor involved in the education of students who will 
work in the real world ofinformation Systems would be: 
1) to point to sources of information, vocabulary, 
rhetoric, methodologies, and appropriate symbolism that 
are considered legitimate (or are valued) in relevant 
social groups; 2) to act as a linguistic guide who would 
sensitize students to the importance of professional 
vocabulary, rhetoric, and other appropriate symbolic 
communication; and 3) to provide students with an 
opportunity to practice and improve their application of 
symbolic knowledge. 

Based on the ideas presented above regarding the 
different types of knowledge relevant in ISD (including 
implementation), we concluded that any ISD class 
should be designed in a way so as to incorporate all of 
the above types of knowledge in a balanced manner. 

Stages of Learning 

We also recognized that learning is not a one shot 
phenomenon, but actually involves a series of 
analytically separable steps. While there are anumber of 



models on learning, for this paper, we adapt Rockart and 
Scott-Morton’s “general learning model (1975, p. 20) 
that was proposed to help understand the learning 
process in higher educational institutions. 

According to this model, learning occurs in four 
progressive stages: 

Acquisition in this stage, the learneris exposedto 
(and asked to remem ber) basic facts and definitions. 

Embedding in this stage, the student thinks 
through facts, skills, and processes learned, and 
applies what he/she has learned in simple 
applications. 

Integration in this stage, “the studentmoves from 
the rote acquisition of material to its incorporation 
and subjugation into more global conceptual 
structures (p. 21). 

Testing in this stage, the student tests the 
- implications of what is learned in new and realistic 
contexts. This also providesan opportunity to revise 
the theories of action the student may have 
developed (Argyris and Schon, 1978) in the earlier 
stages o f learnin g. 

We could see that most ISD classes in universities (at 
least the ones that w e have atten ded) do n ot conscio usly 
lead (and in most cases, do not even attempt to lead) a 
student through the four stages of learning suggested in 
the “general learning model. We wanted to make a 
deliberate attempt to incorporate all four stages in our 
course. 

To summarize our discussion, having recognized the 
needfor a more comprehensive view on ISDcourses, we 
wanted to design a course that would span the different 
types of knowledge (facts anddefinitions, knowledge of 
technology and techniques, propositional social 
knowledge, conceptual social knowledge, and symbolic 
social knowledge) and the four stages of learning 
(acquisition, embedding, integration, and testing). 

COURSE DESIGN 

Objectives 

Specifically, the design objectives for our ISD course 
were the following: 
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Expose students to the complexity of real-world 
systems development and implementation. 

Illustrate the socio-technical nature of information 
systems techniques and technologies based on 
computer sciences, anddesign principles/guidelines 
based on social sciences 

Adopt a balanced approach towards the various 
forms of knowledge and the different stages of 
learning 

Course Design 

The ISD-related courses that we designed at our two 
universities consisted of lectures, assignments, and a 
virtual team project. The lectures and assignments were 
modeled on the traditional modes of teaching ISD 
concepts and skills, focusing primarily on facts and 
definitions, knowledge of technology and techniques, 
and propositional social knowledge. Also, the lectures 
and assignments were gearedtoward the first two stages 
of the learningmodel (acquisition and embedding). The 
virtual team project had two main goals: 1) to help the 
student pass through the remaining two stages of 
learning (integration and testing) regarding technology 
and techniques by providing a real context for 
application that was not artificially bounded, and 2) to 
simultaneously facilitate the embedding, integration, and 
testing of social knowledge, especially conceptual 
knowledge and symbolic knowledge. 

It is useful to point out at this stage that, while in 
principle, the traditional face-to-face team projects can 
also achieve the two goals stated above, we believe that 
virtual team projects are likely to be more effective in 
achieving the goals. The time/space separation in a 
virtual context results in the lack of shared norms 
regarding collaboration among the team-members and 
interaction with the client Consequently, complexities 
arising from social factors tend to get magnified, and 
thus, a majority ofteam-members are forced to reflect on 
the issues, and learn to creatively apply social 
knowledge to solve or at least mitigate the problems. In 
contrast, in face-to-face teams developing systems 
having information requirements of comparable (low) 
complexity, many of the social complications do not 
surface or can be dealt with intuitively by team-m embers 
without deliberate application of social knowledge. 
Thus, students working in face-to-face teams for their 
term projects, focus primarily on the application of 
technology/techniques related knowledge, and fail to 
appreciate the importance of different types of social 



knowledge (especially, symbolic knowledge) that could 
be extremely valuable during the development of large 
scale information systems for complex organizations 
(with different stakeholder groups who do not have 
shared frame of reference) or when working in globally 
distributed teams (Jarvenpaa and Leidner, 1999) for 
systems development. 

In fact, in ourexperience, traditionalface-to-face student 
team-members are farmore concernedabout otherteam- 
members not doing their share ofwork (a concern that is 
less important in the “real world ), than about 
understanding social issues such as user-analyst or 
analyst-developer relationship s/communication patterns, 
developing/clarifying systems specifications for 
individuals not having a shared frame of reference, and 
developing strategies for coordinating and managing 
relationships among members of the ISD team. 

In the rest of this section, we provide a description of the 
virtual team project. We focus on the project, since it 
was our primary vehicle for rectifying (what we saw as) 
the limitations of tradition al cours es in ISD . We felt that 
a virtual team project would be helpful in reflecting the 
reality of today’s geographically separated systems 
development teams. Also, because of the time-space 
separations among team-members in this project, they 
would be forced to confront problems of time 
coordination and lack of shared frame of reference, 
which would, in tu rn, enable them to acquire/internalize 
social knowledge (especially conceptual and symbolic 
knowledge) in addition to the other forms of knowledge 
that they would acquire through more traditional 
mechanisms such as lectures and assignments. In 
addition, since students would be building (over the 
entire semester) prototypes for real businesses with 
actual information problems that did not neatly fit into 
the textbook templates, they would have to go through 
the latter two stages of learning over the course of the 
project. 

In our specific context, the virtual teamswere comprised 
of students from two different North American 
universities working collaboratively to formulate a 
business information systemsproblem, converting it into 
a (database) systems design, and then developing a 
working prototype. More specifically, the participants 
were students enrolled in Information Systems courses 
at two large North American Universities, one based in 
Canada (which we call UA) and the other in the U.S. 
(which we call UB). Typically, each virtual team 
consisted of 4-5 "internal" or "local" group members 
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who were matched with 4-5 "external" or "remote" 
members from the other university. Thus, each virtual 
team consisted of about 8-10 members drawn almost 
equally from UA and UB . The projects lasted for about 
14 week-long semesters. 

The task assigned to each virtual team was to define, 
design and develop an information system for a "real" 
organization. The UA members of each virtual team 
acted as “business systems analysts and were 
responsible for going into an organization and 
identifying a “problem situation that they believed 
needed resolution using a computer-based information 
system.^ They were then supposed to create a rich 
narrative of the problem in the form of an “information 
requirement document (IRD) which had to be then 
transmitted to their counterpart external team members 



at UB who were acting as “systems designers. ^ In 
addition, UA members were required to provide a 
preliminary design of the user-interfaces that would be 
preferred by the organizational clients. The UB 
members ofeach team were then responsible forcreating 
a detailed systems design (including an Entity- 
Relationship or E-R diagram) and a working prototype 
of the database system. Finally, UA and UB team- 
members were to make a Joint presentation (using 
videoconferencing technology) of their entire project, 
including the working prototype. In Table 1, we list the 
participants in this project. Next, in Table 2, we 
summarize the different events of the project with the 
associated time-lines. 



TABLE 1 

PROJECT PARTICIPANTS 



Main P articipants 


Brief description 


UA members 


Members of virtual teams who were students atUA. 
Primarily involved in interacting with the clients and 
defining information and end-user interface requirements 


UB m embers 


Members of virtual teams who were students at UB. 
Primarily involved in logical design and implementation of 
the system based on specifications created by UA members 
in their teams. 


PA 


Professor facilitating the virtual teams from the UA side 


PB 


Professor facilitating the virtual teams from the UB side. 


Companies (each team interacted 
with a different company) 


Located in the sam e city as UA . UA m embers interacted w ith 
company representatives to define the systems requirements. 



TABLE 2 

FORMAL PROJECT STRUCTURE 



Phase of the Project 


Timeline 
(in weeks) 


Event Descriptions 


Phase I: Formation of 
the team and creation of 
work plans 


Week 0 
to week 4 


Event 1: creation of the virtual team by PA and PB 
Event 2: Selection of organization by UA members for 
which the virtual team would develop a system 
Event 3: introductory videoconference #1 
Event 4: completion of project proposal by UA members 


Phase II: Defining the 
business problem 


Week 5 to 
week 8 


Event 5: Completion of the Information Requirements 

Docum ent (IRD) by U A mem bers 

Event 6: Videoconference #2 to clarify the contents of 

IRD 


Phase III: System 
design, development 
and delivery 


Week 9 to 
week 14 


Event?: Completion of conceptual/logical design byUB 
members 

Event 8: Completion of user interfaces by UA 
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Event 9: Prototype delivery by UB members and joint 
presentation in videoconference #3 



BENEFITS DERIVED FROM THE PROJECT 
Perceptions of Students 

The following representativequotations of students after 
completing the project provide evidence regarding the 
benefits that students perceived (emphasis added), 

QUOTATION 1. Overall this was a true learning 
experience. Our group got to see what it takes to 
complete a project with a group thousands of miles 
away. fVe learned that time management was essential, 
and learning about team members* background early 
in the project is crucial in completing the project. Some 
technical skills that we learned were ICQ, Access, 
Database coding, interactive communication through 
videoconferencing and web^board. We discovered that 
each culture has its own customs and one must adapt 
to those differences if the group is to work well. The 
most important thing we learned was communication 
skills. We learned that communicating effectively with 
your (virtual) counterparts is crucial, and if 
communication breaks down then there is little chance 
for success. Fortunately for us, we were able to cross all 
cultural and communication barriers and put it all 
together through team effort. 

QUOTATION 2. I think that as companies grow and 
expand throughout the world, the use of virtual teams 
will become vital. The concept of using virtual teams is 
highly beneficial as it prepares us to deal with other 
people in other loca tions. I learned that MS Access is a 
powerful tool. I learned to manipulate it through SQL 
and through Visual Basic. But most importantly,! think 
I learned to deal with other people in multiple contexts 
and in different situatio ns... this project enabled me to 
see that cooperation is the key ingredient to a 
successfu I project. 

QUOTATION 3. In my opinion, the experience of this 
project has been outstanding. The collaboration really 
gave exposure to the workings of a **real world** 
project, from start to finish, with all the technical, 
organizational, and human challenges one would 
expect in a business scenario. 

QUOTATION 4. At first, I was hesitant to accept that 
this local-remote idea would be beneficial to the project 
but after completing the project, I realize it has been. We 
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have learned database systems and we have also gained 
an unusual and unique experience. We have created a 
database system and application without ever meeting 
(in person) the customer. I have learned many technical 
lessons. Primarily , howto produce a product that meets 
the expectations and requirements of someone else. In 
most class projects that I have worked, we have had to 
develop our own project. Developing a project for a 
remote group mimics the real world much more 
accurately. 

Our (The Professors’) Interpretation 

The four quotations capture a number of lessons that 
students took away from the course. Most importantly, 
the students had an opportunity to integrate and test their 
knowledge about technology/techniques (SQL, VBA, 
EER Modeling, DSS design, etc. that were covered in 
the lectures) in a new real-life context, where the 
problems did not necessarily match the structure of the 
examples covered through lectures and assignments. 
This allowed the students to become significantly more 
competent in the technical domain. 

Another important issue that is reflected in virtually 
every quotation is that the students could appreciate the 
socio-technical nature of ISD, Such appreciation is 
extremely difficult to attain Just through lectures and 
assignments, or by stating the importance of 
sociotechnical thinking by “rote in propositional form. 
The opportunity to innovatively “try out some of the 
conceptual knowledge contributed to the internalization 
and enrichment of the students’ conceptualschema, thus 
advancing a student from a novice towards becoming an 
expert. 

Similarly, the students participating in the project could 
comprehend the importance of dealing with gaps in 
communication, culture, andcontexts, the lack of shared 
frames, and complexities in working across time zones. 
Their comprehension was considerably more profound 
than what could be expected from their attending 
lectures where an instructor droned “propositions such 
as “Communication is necessary for successful ISD. 

Finally, the level of competence in the use of symbols 
gained by participants is beyond question, and this is 
evident from their careful use of “evocative ^ and 
“referential ^ symbols (Couch, 1996). Students also 
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appeared to have realized the importance of building 
team norms for guiding argumentation and other forms 
of communication through the life-time of the project. 

On a more practical note, we received higher teaching 
evaluations(and/or more positive comments) for course 
offerings that involved the virtual team project, which 
reflected higher students satisfaction with the learning 
process (and perhaps, partially, the novelty of the 
experience). Students were exposed to a number of 
communication technologies and formed long-term 
professional networks with local members, and in some 
cases, with remote members. Also, students who 
participated in our project had something interesting to 
say during campus job interviews. For example, in 
response to an interviewer’s question “Can you tell me 
about any challenging project you have done in your 
MIS classes? students were able to enthusiastically 
respond and narrate their experiences and lessons 
learned. In fact, we have received very positive feedback 
from recruiters who visited our universities. For 
example, the lead recruiter of a reputed insurance 
company in the US, sent the following e-mail to one of 
the co-authors after interviewing students who had 
participated in the project: 

... One of the interview questions I always ask is 
what the student's favorite class is. Almost 
every [university name] person I interviewed 
mentioned your [course name] class. That 
speaks volumes for the things you are doing 
with your students and I'm very much looking 
forward to seeing some of this first hand on my 
next visit. 

Not surprisingly, some employers have sought and 
actually hired our students based on (primarily) this 
unique experience of working in virtual teams. 

POTENTIAL PITFALLS 

We hope that the reader would agree that there was 
significant value addition achieved through the IT- 
enabled change in the way the ISD skills were taught as 
part of our “teaching experiment. However, we hasten 
to add that benefits came at a cost. The project drained 
resources (our research time, as well as department 
funds used for travel and videoconferencing). In 
addition, problems encountered in the design, 
development, and implementation were open-ended; thus 
the professors as well as the teaching assistants had to 
work much harder and be more closely involved with 



each team’s project. Sometimes technologies did not 
work. For example, the videoconferencing equipment 
did not function forseveral hours on a Saturday with all 
students waiting impatiently for their team’s turn. The 
likelihood of students being frustrated was extremely 
high, and this put severe stress on us, the professors, 
who, unfortunately, have to sometimes worry about 
teaching evaluations. Students appeared to getfrustrated 
with the facts that: 

Videoconferences were held on Saturdays, due to 
schedule conflicts of different students/facilitators, 
and limited availability of videoconferencing 
facilities. 

The local and especially remote members were often 
perceivedto be unresponsive anduncooperative, and 
technologies were not available all the time (e.g.. 
Web-board was down for few hours). 

Sometimes, from the point of view of the UB 
members, the UA team-members, in their effort to 
impress the organizational clients, provided UB 
members with unrealistic specifications and 
deadlines. Likewise, UA team-membersfelt that UB 
members were technically inept, and were not 
putting in their best effort. 

There was limited guidance available from the 
teaching assistants regarding technical problems 
encountered in implementing some features 
requested by users 

Remote team-members had different/incompatible 
schedules (In fact, we deliberately did not align the 
UA and UB course structures completely in orderto 
retain some of the complexities of real-world 
collaboration between two companies with different 
goals, priorities, and schedules). 

There were uncertainties with respect to the outcome 
of their p roject (and grades). 

Finally, with all the stress experienced by students, 
teaching assistants, and the facilitators (i.e., the co- 
authors), there was always a possibility of abreakdown 
in relationship among the different parties involved, 
especially among the three co-authors coordinating the 
project. While we were fortunate in this regard (i.e., we 
did not experience breakdowns amongst us), and 
received excellent (much higher than usual) teaching 
ratings and seemed to have earned the respect of many 
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perceptive students, we do feel that colleagues 
considering such projects should be aware of potential 
problems and develop contingency plans as part of the 
project design to mitigate negative effects if things do go 
wrong. 

CONCLUSION 

We believe that our model for teaching ISD using 
lectures, assignments, and virtual team-projects can be 
effective in delivering a course that holistically 
incorporates the types of knowledge and stages of 
learning. We have designed a framework^ (Appendix I) 
based on the theory discussed in this paper. We hope to 
refine it and convert it to an instrum ent that can facilitate 
the evaluation of effectiveness (as perceived by the 
students) of the different elements of ourlSD course in 
Spring 2001 the semester when we plan to run this 
project next between a university in the US and a 
Scandinavian university. In other words, we would like, 
as part of our next step, to “objectively validate our 
claim that our course design actually integrates the 
different types of knowledge and encompasses the four 
stages of learning. In addition, we would like to evaluate 
the incremental contribution of the “virtual team project 
over a traditional ISD “team project (based on the 
experiences of a “control group in the following 
semester when we do not run the “virtual team project ). 

To conclude ourpaper, we would like to enthusiastically 
invite members of the lAIM community to incorporate 
similar strategies in their teaching, which we think can 
be very rewarding and fulfilling to the teacher aswell as 
the students. However, we must add a strong word of 
caution the project requires a great deal of planning 

and effort in order to have a chance to succeed. In 
addition, absolute trust and respect (and perhaps 
friendship) is needed among the professors facilitating 
such complex projects to manage the ups and downs. In 
the absence of trust and respect, the project is likely to 
be run into trouble during the term, with disastrous 
consequences in terms of the concerned faculty 
members’ reputation and teaching evaluations. 

ENDNOTES 



31^BSEvSB3^B5NGURG3^^ 

DATABASE SYSTEMS. 

4. Evocative symbols are directed towards developing 
social solidarity. 

5. Referential symbols denote objects, events, and 
sequences referred to during communication and are 
useful in coordinating communication processes. 

6. The framework is an enhancement of Rockart and 
Scott M orton’s “Learning M atrix. 
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APPENDIX I 

A FRAMEWORK FOR EVALUATION 



Type of knowledge with examples 


Role of theprofessor (and IT) 


Component of 
class design 


Acquisition 


Embedding 


Integratbn and 
generalization 


Testing in new 
situations 


FACTS AND DEFINITIONS 
What is a DBMS? 

Define generalizaiion/specialization 

State three advantages of the data- 
oriented approach. 

A relation is in BCNF if every 
determinant is acandidate key. 


Professor provides information. 

The Role of IT is to aid in infor- 
mation disseminatbn. 


Lectures 










Individual 
assignm ents 










Team-project 










“Virtual team 
projects 










KNOWLEDGE OF TECHNOLOGY/ 
TECHNIQUES 

When you wish to represent a 
relationship between a many-to- 
many relationship and another entity 
or relalbnship, use aggregation. 

To transform a 2NF relalbn to 3 NF, 
remove the transitive dependencies. 

The PERT and GA NTT c harts can be 
drawn in Microsoft Project in the 
following way... 


The role of a professor is to “teach 
a student how to use the technology/ 
Technique, and illustrate it w ith 
paradigmatic examples 

The role of IT is to serve as the 
scratchpad for students to try out 
differentsolutions and improve their 
understanding 


Lectures 










Indi vidual 
assignm ents 










Team-project 










“Virtual team 
projects 










SOCIAL KNOW LEDGE 
PROPOSITIONAL 

Communication leading to mutual 
understanding is the key to infor- 
mation requirements analysis 

A prerequisite to successful database 
design is to develop die conceptual 
model based on the views of the 
differentstakeholders On the external 
schema) 

Communication leading to mutual 
understanding is the key to suc- 
cessful project management 


The role of the professor (an 
“expert ) is to articulate these ideas 
clearly and help students remember 
the propositions (as responses to 
certain stimulus) 

The role of technology may be to aid 
the dissemination and periodic 
testingofthe propositions learnedby 
students. 


Lectures 










Individual 
assignm ents 










Team-p rojects 










"Virtual team 
projects 










SOCIAL KNOW LEDGE 
CONCEPTUAL 

The Leavitt's diamond model of 
orgunization;Social construction; the 
hermeneutic circle; metaphor of 
"text to understand action, etc. 

Expanded aspects of propositional 
knowledge 


The professor a cts as a "facilitator 
in exploring abstract concepts and 
theoriesjointly with the students, and 
helping to create (unique) skeletal 
non-deterministic conceptual schema 
in students' minds. 


Lectures 










Individual 
assignm ents 










Team-project 










“Virtual team 
projects 










SOCIA L KNOWLEDG E SYMBOLIC 

Participation in a real virtual team 
systems developm ent project. 

Using videoconferencing (novelty) 

Skills related to readng and 
displaying comm unication s ymbols 


A professor's role is to act as a guide 
in the use of language and other 
symbols he/she sens itizes studen ts 
to political realities of the workplace; 
also acts as an organizer o f a forum 
for students to practice their 
“professional vocabulary and rhet- 
oric in presentations/ argumentatbn. 


Lectures 










Indi vidual 
assignm ents 










Team-project 










“Virtual team 
projects 
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ABSTRACT 

We investigate issues of integration with regard to project man agement and limited course curricula. In particular, 
we note that the evolution of fast-track professionalmaster degrees in information Systems has had a profound impact 
on the quantum, in terms of relevant subject matter delivery, that the instructor that can reasonably impart in a 
focussed environment. We therefore reflect on viable alternatives, id est other subjects, and note that there is 
considerable professional support for subject integration in IS, both in a holistic sense and with regard to project 
management. We argue that this integration could be achieved by modestly realigning relevant subjects in order to 
cover overlapping areas thereby improving on completeness of delivery. 



INTRODUCTION 

Project management is regarded as an essential 
component in theinformation Systems curricula. Indeed 
most undergraduate/postgraduate IT/IS courses offer 
industry-taigeted project management courses. While 
this is fine in itself, we should ask ourselves whether 
what we are teaching is of any benefit to the student and 
future empl oyers. Can we really afford to let them loose 
into “the real world and expect them to come up to 
speed in minimum time with such focussed, discipline- 
oriented approaches? 

Most authors seem to be in agreement that project 
management is really the culmination of the interaction 
between several disciplines (D’H erbemont, Cesar 1 998, 



Lientz,Rea 1 998, Meredith, Man tel 2000). A successful 
project manager realises this and is able to draw from a 
variety of learned skills. A clear and readily accessible 
example can be seen at the Systemation website, where 
project management is broken down into various topics 
each requiring separate tuition. 

Information Systems majors have a particular advantage, 
since they usually enjoy a closer synergy w ith business- 
oriented faculties than other majors. Usually they are 
taught in business faculties where a a good deal of 
supporting material (in terms of peripheral subject 
matter) is readily available. 

The authors have taught project management for many 
years, in different environments, and for different 
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expected outcomes. We note, however, that since the 
introduction (in Australia at least) of fast paced MIT, 
MIS and IT/IS vocational degrees, project management 
seems to have become a “child lost in the wilderness . 
Students emerging from a “one subject encounter 
(amounting to about 24 to 28 hours of lecture-time and 
12 to 14 hours of tutorial/lab-time) with project 
management cannot be expected to possess an industry 
useful level of understanding of the topic. Moreover, 
they cannot also be expected to perform effectively in 
the area unless extensive on-the-job training is 
undertaken in the industry. 

There is a way however to salvagethe situation given the 
correct set of circumstances. We suggest that “the 
project management subject should be employed as a 
focal point bringing together the material learned in 
previous subjects and using it to learn how to manage 
projects. We call this approach the project management 
focus. Forthis reason alone, projectmanagement should 
be taught in the final semester of the final year, so that 
the student and the instructor can draw relevantmaterial 
from as many subjects as possible. Such an approach 
would lead, naturally, to a problem based learning (or 
action learning) instructional design; however, a 
discussion of this is outside the scope of this paper. 

Although projects may have been managed for many 
millennia, it is generally considered that the scientific 
management of projects had its genesis in the 1960s 
Polaris submarine missile project. From this early 
Operational Research/Management Science (OR/MS) 
conception, projectmanagement mainly developed as an 
offspringof engineering and OR/MS . Over recent years, 
however, the concept of project management has 
evolved into an art and a science, has pervaded many 
human activities, and WkQ system the word has 
taken on social comm onsense usage. (For example one 
now finds sections in text books attempting to 
semantically differentiate between social programmes 
and projects, organisations structured as project based, 
and debate in the literature on what constitutes ^project 
and whether there is a difference between project 
management 2 ind managing proj ects .) The importance of 
successful IS project management and the resulting need 
for a wide range of skills is heightened by the challenges 
today’s organisations face. 



ORGANISATIONAL CHANGE, 
CHANGING APPLICATIONS 
AND EMERGING TECHNOLOGIES 

The ability to change is vital for organisational survival. 
The increasingly wicked nature of the environment 
results in erratic and discontinuous change (Drucker 
1969) forcing organisations to constantly adapt. 
Increasing competition, globalizationandthe demand for 
customer focus are some of the forces behind this need 
for change. In addition IT can act as driver, enabler or 
supporter of organisational change (McKeen, Smith 
1996, Oden 1999) and has a profound effect on 
organisations (Lucas, Baroudi 1994). Much of the 
literature focuses on IT rather than IS and in today’s 
organisations IT is perceived to be the base technology 
for IS. In an organisational context statements 
concerning the effect of IT are often also applicable to 
IS. The link between technology and organisationaltasks 
and structure is well established (Leavitt 1964). Changes 
in organisation relevant technologies can bring about 
changes in the structure and function of organisations 
(Groth 1999). IS can blur organisational boundaries 
(Gerstein 1987, Oden 1999) and result in new 
organisational designs. Organisational change should 
therefore be considered as a component of effective 
systems development (McKeen, Smith 1996). The 
effectiveness of the project management process is 
critical for success (Gerstein 1987), since IS project 
failure can have serious consequencesfor organisational 
competitiveness and survival. Pressures on project 
managers to successfully “manage projects are 
therefore mounting. 

IS development takes place in an organisational context 
and may therefore also be affected by events taking 
place in its environment and that may be beyond the 
control of project managers (Mumford, Macdonald 
1989). Organisational change can for example, cause 
problems with participation in IS projects (Reponen 
1998). As organisational boundaries become more fluid 
various groups of stakeholders may be involved in a 
project. Moreover the development of intra- 
organisational systems may include external as well as 
internal stakeholders whose requirements have to be 
considered. While on the one hand organisations form 
loose alliances supported by EDI and Internet 
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technologies, others aim to achieve competitive 
advantage through consolidation. Researchers not only 
advocate that IS professionals and users should work 
closely together, but call also for the integration of IS 
development and HR (Tentenbaum 1999, Oden 1999). 

Concepts developed as a way of helping organisations to 
cope with the challenge of change such as knowledge 
management also draw IS projectteams, users and other 
organisationalpersonnel closertogether(Alavi, Leidner 
1999). Researchers even suggest that in IT-enabled 
network organisations many functions traditionally 
carriedout by IS/IT professionals will shiftto users, who 
will take an active part in IS development (Mutsaers et 
al. 1998). As a resultunderstanding ofhuman behavior 
and learning capabilities isbecoming more importantfor 
successful IS development. Increasingly the boundaries 
between IS and other organisational functions are 
disappearing. This indicates that the integration of 
project management with other subjects may be vital 
when preparing students for the “real world. 

Gradually management’s role is becoming one of 
coordination rather than directions and control (Brooke 
etal. 2000). There is a growingneed for flexibility rather 
than controllability and predicability (Highsmith 1999). 
At the same time IS project management is moving 
towards collaborative management of geographically 
dispersed and fluid teams. Collaborative project 
management is culturally very differentfrom traditional 
projectmanagement (Cutter 2000) and requires different 
approaches, people and skills. IS development teams 
may work and exist in a virtual environment, where 
teams need differentcommunication techniques andthe 
managers require different management skills (Watson, 
Sridhar 1998). 

During the past decade the users and uses of IS have 
changed significantly (Friedman 1993). The focus has 
shiftedfrom IS for organisational support activities such 
as transaction processing to applications for managers 
and professionals and groups rather than individuals. 
There has been a gradual shift in project management 
thinking over the past twenty years from a systems 
engineering /OR/MS focus to one where the focus of 
managing IS projects is more the ability to effectively 
manage complexity, uncertainty, and equivocality than 
IS/IT technical ability. For example, Applegate, 
McFarlan, McKenney (199 7:624-639) suggest in some 
cases a non IS/IT profession al should be selected as the 
project manager; and the traditional project planning 



and control tools [e.g., PERT, milestones, system 
specification standards, change control disciplines etc] 
are less important than contemporary management and 
organisational communication devices, personnel 
controls, and team integration procedures. A study by 
Lee by Lee, T rauth, and Farwell (1995) extends this 
viewto the critical skills and knowledgerequirements of 
IS professionals in general. 

Technology is changing rapidly and new applications are 
emerging all the time. These changes should here fleeted 
in training the IS graduate. This includes areas such as 
the competitive and strategic use of IS, project 
management, change management and collaborative 
work (Gorgone, Gray 1999). Emerging technologies 
very often not only replace old ones but also bring with 
them new methodologies of application that impact 
directly on firms at the procedural, policy and socio- 
technical interface. The external job market for IS 
graduates has changed drastically and it is therefore 
critical that university curricula are adapted to cater for 
these needs. 

PROJECT MANAGEMENT 

Despite the relatively long history of and the laige 
amount of literature available on projectmanagement, on 
the subject of course content and how IS project 
management should be taught in higher educational 
institutions, the literature surveyed by the authors, is 
largely silent. Numerous articles by management, users, 
practitioners, and academics, however, suggest a need to 
re-think the field of IS/IT project management in order 
to learn from the successes and failures. Moreover, from 
a practitioner perspective, studies by Lundeberg ( 1 993) 
show managers perceive IS projects to be always over 
budget and time; and to fail to deliver on expected 
benefits. Lundeberg comments that this may or may not 
be true, but it is management’s perceptions and therefore 
it is something we need to address. The authors consider 
that this re-thinking should include institutes of higher 
education. 

Stohr (1995) states the importance of developing, 
evaluating and re-developing curricula contentto ensure 
that courses rem ain approp riate to the needs o f students 
and employers in the rapidly changing IS/IT arena. 
However, following Toohey (1999) this is the third step 
in a typical course design model 
(see Figure 1). 
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FIGURE 1 

A TYPICAL MODEL OF THE COURSE DESIGN PROCE SS - ADAPTED FROM TOOHEY (1999:21) 



1 2 3 .4 




The needs and demand for a course in IS/IT project 
management is essentially acknowledged and indeed 
supportedby professional associations such as the ACM, 
ACS, AIS, etc.. These bodies are often initiators of 
events that lead to eventual curriculum changes, and 
moreover provide knowledgeable input to curriculum 
design. For example, project management is a core 
requirement for gaining course accreditation with the 
Australian Computer Society (ACS). This is supported 
by the overview report of the first joint curriculum 
development effort for undergraduate programs in 
information systems (Couger et al. 1 995) which includes 
the subject w ithin the IS degree structure; and a study by 
Lee et al. (1995) which showed management skills 
(including managing projects) to be critical to the 
development needs of IS professionals. 

In the absence of academic literature debate, the authors 
turned to the special issue on IS curricula and pedagogy 
(MIS Quarterly 19(3); in particular Couger et al. 1995) 
and to a survey of project management textbooks. On an 
undergraduate project management subject, Couger et al. 
(1995:351) state: 

Scope: This course covers the factors necessary 
for successful management of systems 
development or enhancement projects. Both 
technical and behavioral aspects of project 
management are discussed. 



Topics: Managing the systems life cycle: 
requirements determination, logical design, 
physical design, testing, implementation; system 
and database integration issues; metrics for 
project management and systems performance 
evaluation; managing expectations: superiors, 
users, team members, and others related to the 
project; determining skill requirements and 
staffingthe project; cost-effectivenessanalysis; 
reporting and presentation techniques; and 
effective management of both behavioral and 
technical aspects of the project. , 

Couger et al. (1995) do not develop this curriculum 
content furthernor do they explain how this content can 
be shoe-horned into a single (or double) semester unit. 
They do, however, note the prerequisite need for IS 
students to have a “foundational knowledge in the 
disciplines within business administration ; and to have 
people skills, an understanding of oiganisations 
(functionsand behaviours), and a basic understanding of 
mathematics, and the behavioural, social, and natural 
sciences(p.347). While this outline content, potentially, 
indicates a balance between the technical and people 
skills needs of a contemporary project manager the 
curriculum of the IS Major appears to be largely 
focussed on the technical skills, ab ilities, and knowledge 
required of an IS professional, (cf Applegate et al. 1997, 
Lientz, Rae 1999, D’Herbemont, Cesar 1998, Shtub et 
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al. 1994, Badiru, Pulat 1995). Indeed, the “Represent 
tative Abilities and Knowledge for iS Program 
Graduates presented by Couger et al, (1995:346) are 
firmly based in an epistemological and ontological world 
view of social objectivism that is: functionalist 
sociology (Burrell, Morgan 1980, Hirschheim et al, 
1995), From an IS perspective, Hirschheim etal. (1995) 
and Lundeberg (1993) suggest this world view is 
reflectedin the principal IS development methodologies 
and methods of the 1970s and 1980s. Such an approach 
may on longer be adequate in an era of persistent 
organisationalchange, sophisticated and influential users 
and new technologies, 

THE NEED FOR INTEGRATION 

One of the proposedtraits in IS graduates put forward by 
the ACM/AIS on IS Graduate Curricula (Gorgone,Gray 
1999) is the need for a strong integration of IT with 
business, a core knowledge of IS and most importantly, 
those specific skills leadingto a career path. One of the 
sample career tracks put forward in the proposal is 
Project Management. The task force explicitly 
recognises that (Gorgone et al. 1998): 

...project management is an important topic for 
the core at the M S level because it is a capability 
that an MS degree guarantees. ...Almost all IS 
development involve transforming an 
organisation from its existing ways of doing 

things Students should understand and be 

able to implement the changes thatan IS system 
creates. 

An interesting and important recognition made by the 
ACM/AIS task force is in line with the proposal of the 
paper (arrived at independently in both cases) that is, 
integration in terms of the interrelation ship between 
courses. It therefore follows (from this generalisation), 
that project management as a discipline and career path 
would culminate in the final year with the integration all 
of those aspects that make Project Management a 
discipline. This aspect has been recognised by various 
organised professional bodies, so much so in fact that 
industry and now some universities (at the graduate level 
at least) are offering exclusive courses in the discipline. 
Further, the ACM/AIS task force argues that some 
aspects of Project Management and Change 
Management are dealt with in indirectly in other parts of 
the proposed curriculum. The authors argue that 
although this is an important recognition, the task force 
does not go far enough in identifying all of those 



elements that could be factual in the project management 
career track. 

The recognition of curriculum integration is fundamental 
and has direct bearing on the student’s capacity to 
recognise the connection between subject and their 
relevance to practicums, the fact that people do not use 
disciplines in isolation , it empowers critical thinking, it 
prevents curriculum fragmentation, prevents curriculum 
overload, it increases the ability to meet the student’s 
learning needs, it recognises and minimises curriculum 
overlap, etc. (Jacobs 1998). 

Although part of the knowledge needed to manage 
projects is restricted to “project management as seen in 
the traditional sense id est, critical path analysis, 
resource scheduling and so on, there is a great deal of 
overlap with other management disciplines such as 
business/corporate law, risk analysis, finance and 
accounting, planning at the strategic/tactical/strategic 
level, knowledge oforganisationalstructu res/behaviours, 
industrial relation issues, personnel relation issues, 
counselling, conflict resolution, etc. 

As a basis for the argument of integration the project 
management framework proposed by PMI (the project 
management institute) provides a candid view of what 
professional associations see as the areas that 
influence/drive/support project management. The 
project management framework therefore is an ideal 
source of facts from which to b egin an integrated project 
management curriculum design. 

We note that within the PMI project management 
framework are the following (a more detailed listing is 
at Appendix 1): 

1. Project Integration Management Concerned with 

the coordination of the project elements. Of 

particularimportance at this stage is the recognition 
that Project Integration Management is primarily 
integrative. It requiresthe documented development 
of a project plan, the project plan execution, and the 
development of control mechanisms to oversee and 
coordinate change across the project. 

2. Project Scope Management . Defines the 

boundaries of the project in terms of the product 
produced and the work that must be done in orderto 
deliver the product. It consists of the initiation 
phase followed by scope planning, scope definition, 
scope verification, and scope change control. Scope 
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planning requires knowledge and skills in 
product analysis, benefit/cost analysis, 
alternative identification, and expert judgement. 
Scope definition requires knowledge in the area 
of work breakdown structures in order to 
formulate adequate templates forthe distribution 
of work into smaller and easier to manage 
blocks. Scope verification is the step where the 
project scope is presented to the client for 
acceptance. Scope change and controlidentifies 
and introduces those mechanisms which allow 
for the determination, introduction and 
management of beneficial changes. 

3. Project Time Management . Includes those 
processes which are required to ensure the scheduled 
completion of the project. It requires identifying 
and defining each activity to be carried out for the 
completion of the project. 

4. Project Cost Management . Requires those skills 
necessary to ensure that the project is completed 
within the allocated budget. 

5. Project Quality Management . Requiredto ensure 
that the project will deliver a product within its 
defined scope. Key areas identified with quality 
management are quality planning, quality assurance, 
and quality control. 

6. Project Human Resource Management . To make 
the most effective use of the human resource 
involved in the project. 

7. Project Communication Management . Includes 
the processes required to, ultimately, disseminate the 
appropriate information in order to carry out and 
complete the project within schedule and budget. 
The required guidelines for proper communication 
management are communications planning, 
information distribution, performance reporting, 
administrative closure. 



8, Project Risk Management . Is required to identify, 
analyse and respond to project risk. It is concerned 
with risk identification, risk quantification, risk 
response development, risk response control. 

9. Project Pro cu rement Management . It involves the 
acquisition of resources (goods and/or services) 
outside the organisation responsible forconducting 
the project. Areas of interest in procurement 
management are procurement planning, solicitation 
planning, solicitation, source selection, contract 
administration, contract completion. 

Using the data at Appendix 1, the PMI project 
management framework can be represented 
diagrammatically as in Table 1, The Table shows the 
degree of overlap with other disciplines in a typical 
business faculty; and the degree of relative integration 
importance as indicated by the number of mentions 
given to activities in the PMI project management 
framework. Of note is the substantial overlap in areasof 
information systems, management, human resources 
management, industrial relation, accountingand finance, 
economics, industrial relations, and some marketing. 
We note the relative low integration mention of 
organisational development (change management and 
team work etc) and communication-oriented subjects; 
and we are in support of the (Australian) industry view 
that subjects lack quality in terms of imparting general 
critical thinking skills to the students. Here we refer to 
the recent Australian Federal Government survey of 
employers conducted by AC Neilson which found (in 
part) that university graduates needed to improve their 
independent and critical thinking, creativity, oral 
communication, and problem solving skills if they 
wanted to become more attractive in the job market, 
(http:/Avww, abc.net.au/news/newslink/weekly/) (ABC) 
Project managers have traditionally focussed on cost, 
time, and quality. Ignoring for the m ost part people. We 
suggest that, indicatively. Table 1 implies a 
reinforcement of that approach. However, in the current 
(and future) organisational environment, we would 
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TABLE 1 

DIAGRAMMATIC REPRESENTATION OF THE PMI PROJECT MANAGEMENT FRAMEWORK 



Broad Activity 


Relative Mentions 




Providing Section 


Organisational Policy 


X 




X 


xxMngt/Acct/Econ/Mkt 


Corporate/contract law 


X 




X 


xIR 


Marketing principles 


X 




X 


Mkt 


QM/Systems Engineering/QFD 


X 




X 


xxxxxxMngt 


TQM tools 


X 




X 


xxxxxMngt 


Financial analysis/planning/control x 




X 


xxxxxxxxxAcct 


Contingency planning 


X 






Mngt 


Risk classification 


X 






Mngt 


WBS 


X 




X 


xIS/IT 


network modelling 


X 






IS/IT 


estimating 


X 




X 


xxxxAcct/Econ 


MS/OR 


X 




X 


xxxxxxxIS/IT/Math/Econ 


Modelling/simulation 


X 




X 


xxxxxxxIS/IT/Math/Econ 


Scheduling 


X 






Mngt/IS/IT 


management and control systems 


X 




X 


xxxxMngt/IS/IT 


PMIS 


X 






IS/IT 


Information distribution/retrieval 


1 X 




X 


IS/IT 


systems 










Benefit A nalysis 


X 




X 


xxMngt/Acct/Econ 


Performance measurement 


X 




X 


Mngt/Acct/IR/HRM 


Stakeholder needs analysis 


X 






Mngt/Mkt/IS/IT 


Management and Leadership 


X 




X 


xxxMngt/HRM 


theory/skills 










Negotiation/communication 


X 




X 


xMngt/HRM/IR 


theory/skills 










Organisation Development skills 


X 




X 


xMngt/HRM 


Audits/reviews 


X 




X 


xxxxxAcct 


Legend: Acct: Accounting 


Econ: 


Economics 




HRM: Human Resource Management 


IR: Industrial Relations 


IS: 


Information Systems IT: Information Technology 


Math: Mathematics 


Mkt: 


Marketing 




Mngt: Management 
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argue, general management competency and people 
skills (including change management, organisational 
developm ent, communication, and negotiation skills are 
as important as cost, schedule, and quality possibly 
more so as they can bridge, the gap that often exists 
between these more traditional project management 
criteria. Successful project managers recognise the 
importance of people, after all how could we conduct a 
project without people. They also recognise that 
workers play an integral role in completing quality 
projects. Indeed, people are the initiators, developers, 
and deliverers of projects as well the users of projects 
products. We suggest that this view is supported by the 
findings of a joint academic and industry study into the 
critical skills and knowledge needs of IS professionals 
(Lee et al. 1995). That study found: 

...technical specialist knowledge to be the least 
important for both now and in the future. At the 
same time, business functional knowledge and 
interpersonal/management skills were 
considered the most important in the future 
(p.327). 

We support the findings of Lee et al. (1995) and 
intuitively note that these results appear to be in direct 
conflict with Table 1 and the AC Neilson survey results 
(reported earlier). Whatsoever, we caution that we have 
not located any body ofresearch literature to support (or 
otherwise) our intuition. Consequently, we identify this 
as an area for our future research. 

CONCLUSION 

We have suggested that project management delivery 
can be enhanced considerably by using synergies that 
exist in a business-focussed environment. Specifically 
we drew attention to a growing wealth of pedagogical 
material which advocates integration as the means to 
produce a better focussed, critical thinker able to use a 
more holistic approach to problem solving. Further, we 
drew attention to material forwarded and supported by 
professional bodies, highly indicative of the same 
philosophy of thought in the areas of project 
management/projectadministration/change management. 
Finally, we have identified this as a potential area for 
future research. 
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APPENDIX 1 

BROAD PROJECT MANAGEMENT ACTIVITIES AND REQUIRED SKILLS 



Broad Activity 


Skills 


Project Integration 
Management 


Knowledge about work breakdown structures, cash flow 
forecasts, usage of estimatingdatabases, organisational policies 
including quality management, pers onnel admin istration , and 
financial controls, linear algebra with constraints models is also 
applicable. Computer simulation, project management 
information systems, communications, leadership and 
negotiation theory, and finally games and decision theory to 
quantify unknowns. 


Project Scope Management 


Benefit measurement methods in terms of comparative 
approaches, scoring models, benefit contribution and econom ic 
models. Constrained optimisation methods such as 

mathematical models using linear, non-linear, integer algebraic 
methods. An appreciation, at lest, of Marketing principles such 
as market share and public perceptions. System engineering, 
value engineering, value analysis, function analysis, quality 
function deployment, estimating tangible and intangible costs 
and the various benefits from project alternatives, and financial 
measures such as ROI or payback period are needed to assess 
the viability of the identified alternatives. Works breakdown 
structure templates project decomposition to the point where 
adequate costs and duration estimates can be achieved. 
Inspection in the form of reviews , audits and structured- 
walkthroughs are a requirement of this stage. Management 
approval structures, tracking systems and integration with the 
overall change control system. Performance measurement 
techniques are also employed. 


Project Time Management 


Interactivity dependencies and their orderly sequence. 
Estimating the duration of each of the activities against a time 
scale. Creating a project schedule. And finally setting up a 
control mechanism to apply any changes. 


Project Cost Management 


Involves aspects of resource planning, cost estimating, cost 
budgeting, and cost control. Work breakdown structures, 
resource planning, organisational policies, estimation 
techniques such as analogous estimating, parametric modelling, 
bottom-up estimating, computer aided estimates, tracking and 
auditing systems, earned value analysis, tracking systems, 
earned value analysis . 


Project Quality Management 


Benefit/costanalysis, bench marking, flowcharting techniques, 
simulation, quality audits, control charts, pareto diagrams, 
statistical sampling, [familiarity with quality standards?] 
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Project Human Resource 
Management 


Organisational planning staff acquisition, team development. 
Knowledge for carrying outhuman resource management needs 
to be acquired from negotiation skills, team-buildingactivities, 
general management, reward and recognition systems. 


Project Communications 
Management 


Stakeholder information need analysis, communication skills, 
information retrieval systems, information distribution systems, 
performance reviews, variance analysis, trend analysis, earned 
value analysis. 


Project Risk Management 


Flowcharting, risk classification schemes, interviewing, 
expected monetary value, simulation, statistics, decision trees, 
expert systems, contingency planning, corporate law. 


Project Procurement 
Management 


Make or buy analysis, expert systems, contract law, contract 
negotiation, marketing, estimating, tracking systems, 
accounting, procurement audits. 
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ABSTRACT 

This paper examines the assessment of an undergraduate program in Information Systems. It presents an 
approach to program assessment that has proven beneficial to one institution. The data from program 
assessment can be useful to stakeholders and forms a baseline for program improvement. 



INTRODUCTION 

A major trend in U.S. higher education of the last ten 
years has been towards educational assessment. 
Universities, accrediting organizations, professional 
academic associations, and government review panels 
have requested and often demanded assessment 
information from schools, departments, programs, 
projects, and individual faculty. One example is the 
AACSB accreditation standards for business schools, 
which require for accreditation that a variety of 
assessment processes be in place. In the field of 
information systems, not much has been reported in 
the literature on program assessment, yet it is 
important because IS programs are growing in size and 
their success needs to be monitored and informed steps 
taken to improve the curricula. 

This paper offers an assessment model approach for 
program assessment of IS undergraduate programs. It 



presents the case example of results of program 
assessment for the B.S. in Information Systems at 
University of Redlands (UR). It demonstrates the 
usefulness of the approach to understanding the overall 
success of an IS degree program as well as details of 
program elements. The example points to many ways 
that the outcomes of assessment can be utilized for IS 
program improvement. 

The B.S. in IS at UR is a 14 year old program that had 
undergone three major curricula upgrades, with a 
fourth one planned in the fall of 2000. The prior 
curricula changes were effected in the years 1986, 
1992, and 1997. The curricula were based on national 
model curricula, including the DPMA 90 curriculum, 
and the IS 97 curriculum (it was available in 
preliminary form two years prior to its final release in 
1997.) The Redlands B.S. in IS is a degree completion 
program that includes only junior and senior 
undergraduate years, with the first two years of work 
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done at another university or college. The Redlands 97 
curriculum’s knowledge attainment goals are shown in 
Table 1. The Redlands 97 IS curriculum consists of the 
IS major consisting of nine IS courses, as well as one 
humanities course, one upper division mathematics 
course, and four business courses (See Table 2). The 
nine IS courses encompass nearly all of the IS '97 
model curriculum, with the exception of several 
introductory courses. However, since the students 
enter as working adults, many with IS industry 

TABLE 1 

KNOWLEDGE ATTAINMENT GOALS OF BSIS 
MAJOR PROGRAM 

• Understand an apply structured programming 

techniques 

• Be conversant in fundamental C language 

capabilities 

• Understand the Graphical User Interface (GUI) 
principles 

• Be conversant in fundamental Visual Basic 

language capabilities 

• Understand the basic terms and components of the 
relational database model 

• Understand and be able to apply database 

application development using Access in 

conjunction with the VisualBasic programming 
language 

• Understand and apply the fundamental concepts 
and terminology associated with networking and 
telecommunications 

• Understand and be able to build internet and web- 
based applications 

• To appreciate the design and structure of 

geographic information systems as a 

decision_making tool 

• To demonstrate knowledge of decision support 
systems, executive information systems, and 
expert systems including some hands-on 
applications 

• To understand how the decision making process 
can be enhanced by modeling 

• To appreciate the practical benefits of 

management support systems in real business 
settings and in society 

• To obtain skills in spatial analysis 

• To become knowledgeable of the ArcView GIS 
software applications. Be able to implement 
hands-on basic GIS functions using ArcView 

• To appreciate how decision_makers in the public 
and private sectors can better assess and make 



experience and jobs, a more condensed curricular 
introduction was felt to be appropriate. The IS major 
is distinguished by two required courses that are not 
standard in IS '97: a course in geographic information 
systems (GIS) and a course in computer ethics. Both 
of these courses relate to strengths of the university: it 
is known for its strong GIS faculty and it has a liberal 
arts foundation that ethics relates to. 
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decisions on managerial problems through the use 
of CIS 

• Understand the concepts and principles of 
structured and object oriented methodologies in 
systems development 

• Be able to apply effective system modeling tools 
and skills including CASE 

• Understand the perspective of complete life cycle 
activities, their value and application 

• Understand information systems from the 
perspective of pieces of an organizational system 

• Understand the magnitude of technological 
advances and their implications to ethical issues 

• Understand the theoretical perspective of ethical 
philosophies as they relate to business and 
organizations 

• Identify and understand examples of real world 
ethical dilemmas in information systems including 
the many sides of such dilemmas 

• Be able to discuss and properly communicate the 
basic concepts of information systems 

• Be able to design and build a business information 
system 

• Be able to develop and implement software in a 
business project solution 

• Communicate a complete systems project 
including verbally presenting and writing up and 
documenting the project 
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TABLE 2 

CURRICULUM OF BACHELOR OF SCIENCE 
IN INFORMATION SYSTEMS (BSIS) 



ISYS312 Productivity with Information 

Technology 

MGMT310 Philosophical Foundations of 
Management 

ISYS219 Fundamentals of Structured 

Programming with C 

MGMT330 Managing and Leading 

Organizations 

ISYS317 Introduction to Programming 

Techniques: Object-Oriented 

Programming 

MTHW 303 Mathematical Applications 
for Information Systems 

BUAD 337 Political and Business Economics 

IS YS 3 1 8 Database Concepts 

BUAD 469 Accounting Information Systems 

ISYS 425 Management and Decision 
Systems 

ISYS 365 Geographic Information Systems: 

BUAD 340 Principles of Marketing 

ISYS 489 Applied Software Development 

Project II 

ISYS 415 Computer Ethics 

ISYS 404 Systems Analysis and Design 

ISYS 488 Applied Software Development 

Project I 

After the Redlands IS 97 curriculum had been run for 
two years, the IS faculty initiated a plan of program 
assessment that was carried out in early 2000. The 
assessment model consists of the following steps: (1) 
written survey of the 300 current students of the extent 
that their knowledge or competency of the program 
objectives and on their suggestions for program 
enhancements, (2) written survey of 700 alumni of the 
program on the extent that program objectives have 
helped them in their careers and on their suggestions 
for program enhancements, (3) focus group with the IS 
Degree Program Corporate Advisory Committee on 
their recommendations for curricular improvement, (4) 
focus groups with samples of current students about 
their suggestions for curricular improvements, and (5) 
IS faculty review and suggestions of current 
curriculum. 

The approach is multi-faceted, i.e., it includes many 
stakeholders and several methodologies. In particular. 



it gathers data from current students, alumni, corporate 
advisors, and IS faculty. It utilizes the methods of 
written survey, directed focus groups, and discussion. 
All of this information is aggregated together to form 
part of the IS program assessment report. Other 
information required by the university is added to the 
report including for instance data on the program, 
current status of the curricular and support services, 
advising, and math and writing skills. Altogether, this 
larger amount of data than is ordinarily collected 
provides a more in-depth insight into the program 
successes and failures, strengths and weaknesses, and 
feeds information into design of the future curriculum. 

Results of Assessment Procedures 

Program. The BSIS program was assessed through 
three open-ended questions in two surveys, one of 
current students and a second of alumni. A third 
program assessment was conducted through two focus 
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groups with BSIS students. In the hour and a half 
focus group sessions, the students were first presented 
with the results of the BSIS curricular survey. Then 
there was a segment of discussion of the curricular 
results. Finally, there was a 45 minute open group 
discussion of suggestions for the BSIS program and 
curriculum. 

Only the survey responses responses from current 
BSIS students that made specific recommendations are 
included, i.e., a comment such as "great program" is 
not included because it does not inform the assessment 
process. Question 3 pertains especially to the 
program. There are several general points coming out 
of these comments regarding the program. On is that 
student would like to see the latest technology 
emphasized in the program. This implies that in 
program revision, attention needs to be given to the 
most current and important technical skills. Responses 
also support review for very recent and current 
textbooks underpinning the technical skills. There are 
several comments about workshop/tutorial preparation 
in math and writing skills as important. In general the 
administrative mechanics and support services of the 
program are not commented on, which would imply 
that they are working well. 

The summary of the two BSIS student focus groups 
underscore several points regarding the program. One 
is that the pre-entry counseling and advertising 
materials need to correctly position the program 
balance. In particular, the BSIS program offers a 
balanced blend of managerial and technical content. 
However, some incoming students have mis-read the 
program as strictly technical. Students are not clear on 
the level of technical expertise that they will be 
expected to achieve i.e., will they be intermediate 
programmers or master programmers? The program 
needs to articulate its goals early on. One comment in 
response is that the survey numerical responses 
indicate a considerable range of incoming technical 
knowledge/competency in particular between 1 and 
3.5 on a 1 to 5 Likert scale. Hence, part of the 
misunderstanding is that students technical mastery 
levels will remain different throughout the program 
and what can be standardized is the minimum level of 
technical knowledge/competency. 

Overall, the student suggstions/recommendations are 
centered on the technical content of the program; they 
are seeking expanded technical coverage. They appear 
satisfied with the program administration and 



academic support and service elements. 

Curriculum. The survey of current students 
measured Likert scale responses on the extent of 
current knowledge of 26 program objectives. For 
N=53 responses, the average current knowledge was 
2.59 on a 1 to 5 Likert scale, with 5 being the highest 
value. This reflects an intermediate state of current 
knowledge, which is to be expected for the full 
spectrum of students over the two years of the 
program. The comparison of students in the first half 
of the program i.e., junior year to those in the second 
half i.e., senior year shows an average current 
knowledge of juniors of 2.45 and of seniors of 2.73, 
for an average gain of 0.28, or eleven percent. When 
juniors are compared with students who were seniors 
in the last six months or had just completed the 
program, the average gain in current knowledge was 
0.57 or twenty three percent. These results provide 
confirmation of the learning value of the IS major. We 
interpret that the gains are not even higher because the 
student population consists of working adults students, 
most of whom come into the program as IS or 
technology professionals, so their starting knowledge 
base is higher than for traditional younger resident 
students. 

The gains in current knowledge from the program can 
be further disaggregated by the sixteen program 
objectives. Comparing juniors to seniors in the last 
nine months and a few very recent graduates, the 
following are the ranked gains by program objective. 





Current BSIS Students 




Rank 


BSIS Program Objective 


Av, Gain in 
Current 
Knowledge from 
Jr. Year to Last 9 
Mos. 


1 


Access in Development 


1.06 


2 


Dec. Making with CIS 


1.05 


3 


Communicate Project 


1.03 


4 


Database Applications 


0.98 


5 


Relational Database 


0.94 


6 


Methods of Syst Analysis 


0.94 


7 


CASE & Syst Tools 


0.92 


8 


Visual Basic 


0.79 


9 


Structured Programming 


0.78 


10 


Syst Life Cycle 


0.63 


11 


ArcView Software 


0.57 


12 


Internet & Web Applications 


0.56 
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13 


GUI Interface 


0.54 


14 


Communic. Basic Concepts 


0.53 


15 


C Language 


0.47 


16 


DSS/EIS 


0.46 


17 


Decision Making 


0.45 


18 


Design/Build an IS 


0.43 


19 


Spatial Analysis 


0.36 


20 


IS Project Solution 


0.35 


21 


Ethical Dilemmas 


0.33 


22 


Organiz. Systems 


0.31 


23 


Ethical Philosophies 


0.13 


24 


Ethic Implic of Techn. 


0.13 



and in the mainstay areas of data-bases and systems 
development. Also, CASE tools and visual basic are 
high. Those are areas in the present curriculum that 
appear to be working well and are likely to be retained 
in the new BSIS 2000 curriculum although 

modernized. 

On the other hand, areas that are not doing well as 
measured by the extent of knowledge gain are 
networking concepts, management support systems, 
certain aspects of computer ethics, organizational 
systems, and solutions with IS projects. These are 
pointing to areas in the BSIS 2000 curriculum in 
which to redesign courses and educational strategies 
and to seek major teaching and learning 
improvements. For instance, the low score for 
networking concepts may stem from the lack, until 
recently, of a hands-on networking laboratories for 
training on NT administration and configuration. 
Small practice networks have now been located in the 
Jones Center and all the major regional centers. Also, 
the networking course was in need of revision, a task 
which has recently been accomplished, but may need 
even more revision and attention in the new 

curriculum. 

There are a number of gender differences in 
knowledge/competency by program objectives. The 
statistically significant gender differences for 
competency/knowledge of current students are for: 
structured programming and C language (p=0.05) and 
there are lesser differences for networking and ethical 
dilemmas (p=0.10). In all these cases, the knowledge 
competency of males exceeds that of females. The 
programming and networking differences may relate 
to occupational differences prior to entry, while the 
difference in knowledge of ethical dilemmas may 
related to differential prior exposure to ethical 



25 


Mgt Support Systems 


0.10 


26 


' Networking Concepts 


0.04 



The information in the disaggregated table is valuable 
in determining where the most learning is taking place 
and where learning is not as effective, so future 
improvements can be focused in those areas. The 
results confirm extensive learning in GIS decision 
making, a new area for most students, in 
communication of project. 



dilemmas in the real world. 

The knowledge/competency results can be grouped by 
major curricular areas. For instance, the Systems 
curricular area comprises the knowledge/competency 
objectives of methods of systems analysis, CASE and 
systems tools, systems life cycle, designing/building 
an information system, and IS project solution. When 
the knowledge/competency of current BSIS students is 
grouped by major curricular areas (see Figure 1), the 
highest levels of knowledge/competency are in 
communications/organization/ethics, while the lowest 
levels are in DDS/GIS and systems, with technical in 
the middle. This emphasizes that the students have 




range from 0 to 5, but actualy are in the range from 1 to 5. 
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considerable knowledge of the softer sides of the 
curriculum. Figure 1 also demonstrates a gender gap 
overall and for the four major knowledge/competency 
areas. JJ^is mostly relate to the prior levels of 
experience am 
entered the BSIS ] 

have had more expe 

To a less 

differences in s^f 
>» 

Since tl^ gende 
differei^e t^r: 
regards thfe reason 



by motivational factors i.e., that students are more 
motivated to learn about the technical and systems 
areas due for job and career related reasons. 



The ^in in kr 
curricular ^eaa.(Fi^ 
is hipest in 

low^t in the 

2.00 

organizations, and 
learning gain of 23 
curriculum ocdffrs 
student entry know 
curriculum is 



4 

♦ 

♦ ♦ 



♦ 

♦ 



♦ 

♦ 






♦ 

♦ 






moderate to weak? areas initia^lly. Un tn^ other Hand^ 

the least learning ^SIS Start 

communications/organization/ethics. This is the area 
of strongest initial knowledge. This may be explained 



E 1 

COMPETENCY OF 
IRICULAR AREAS, 
IMNI 



FIGURE 2 

OVERALL KNOWLEDGE/COMPETENCY VS. 
TIME IN PROGRAM, CURRENT BSIS 
STUDENTS 
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Correlation analysis for the aggregated knowledge/ 
competency areas and with elapsed program duration 
in months shows signficant correlations among the 
overall knowledge competency and the four important 
areas (see Tables 3 and 4). This implies that generally 
a student is higher or lower in knowledge competency 
in all the areas i.e., they vary up and down together 
among individuals. This points to a more generic set 
of knowledge/competency across areas. This may 
reflect the real world experience of adult students. For 
instance, a student who had worked in a systems 
analysis position for ten years before entering the 
BSIS would have built up knowledge in all of the four 
major areas, while an entering student without any IS 
job experience would be weak in the technical, 
systems, and DSS/GIS areas. It is somewhat less clear 
why he/she would be weak in 
communications/organizations/ethics as well, but that 
is the case. 

The correlations also show the correlation with elapsed 
time in the program. There is a statistically significant 
correlations of overall and technical knowledge/ 



competency with elapsed time, but not with the other 
areas. This is seen in the scatter plot shown in Figure 
2. What is clear in all these areas is that there is a 
large range of incoming competencies; again this 
relates to the population of adult students. For overall 
knowledge (Fig. 1) and technical knowledge, there is a 
significant trend of increase over the two years. Yet, 
at graduation, there is still substantial range of 
knowledge/competency. The trend line has moved up, 
but the high variation remains. This is helpful in 
explaining teaching challenges even late in the BSIS 
program of a large range of competencies in a single 
cluster group. We interpret this as continuation of 
incoming ranges. By contrast, for 

communications/organizations/ethics, the average 
incoming knowledge/competency is higher but with an 
even larger range and doesn’t increase much with the 
same large range throughout the program. 

The data from the alumni survey were equally 
revealing. The key question related to program 
objectives is different; it asked how well the IS degree 
had imj3acted the alumni's career advancement. The 
following were the ranked program objectives (on a 1 
to 5 Likert scale, with 5 being the highest impact): 

The alumni results point to the importance of the 
"softer" sides of the curriculum on career impact — 
areas such as communications, organizations, 
designing and building an IS project, and the "building 
block" areas of systems development and database. 
The top two objectives are the area of interpersonal 
communications. This result corresponds to other 
studies in the IS educational literature that stress the IS 
career advances of 
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interpersonal skills and teamwork (see literature 
review in Pick and Schenk, 1993). In general, this 
literature points to the "softer" areas as ultimately 
having more impact on IS graduates careers in the long 
term. It is important to ask why these behavioral skills 
rise to such importance. One reason may be the rising 

TABLE 3 

CORRELATIONS OF IMPORTANT 
KNOWLEDGE/COMPETENCY AREAS FOR 
CURRENT BSIS 



career attainment of BSIS alumni, so that eleven 
percent of the alumni are in top management 
(President, VPs) and forty percent are in middle 
management or higher. In middle to top management 
jobs, the areas of communications, teamwork, and 
organizations are essential. 



Months Overall Technical DSS/GIS Systems C/O/E 



Months 


1 








Overall 


0.309* 


1 






Technical 


0.349** 


0,923** 


1 




DSS/GIS 


0.273 


0,863** 


0709** 


1 


Systems 


0.251 


0.923** 


0.798** 


0.732** 1 


C/O/E 


0.186 


0.891* 


0.682** 


0.771** 0.852** 



* significant at 0.05 level 
** significant at 0.01 level 

LE4 

CORRELATIONS OF CAREER IMPACT ON 
ALUMNI FOR IMPORTANT BSIS 
CURRICULAR AREAS 



Year Overall Technical DSS/GIS Systems C/O/E 



Graduated 








Year 

Graduated 


1 








Overall 


-0.187 


1 






Technical 


-0.047 


0.818** 


1 




DSS/GIS 


-0.287* 


0.754** 


0.436** 


1 


Systems 


-0,235 


0.817** 


0.579** 


0.519** 1 


C/O/E 


-0.202 


0.746** 


0.351** 


0.560** 0.623** 



* significant at 0.05 level 
** significant at 0.01 level 





Alumni 






BSIS Program Objective 


Impact of BSIS 
Program 

Objective on 

A lumni 's Career 


Rank 




Advancement 



1 


Communicate Project 


4.08 


2 


Communic. Basic Concepts 


4.04 


3 


Syst Life Cycle 


4.00 


4 


Organiz. Systems 


3.98 


5 


IS Project Solution 


3.97 
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6 


Design/Build an IS 


3.92 


7 


Relational Database 


3.84 


8 


Database Applications 


3.79 


9 


Ethical Dilemmas 


3.75 


10 


Ethic Implic of Techn. 


3.73 


11 


Ethical Philosophies 


3.69 


12 


Access in Development 


3.63 


13 


Networking Concepts 


3.53 


14 


Mgt Support Systems 


3.50 


15 


Methods of Syst Analysis 


3.36 


16 


Decision Making 


3.31 


17 


Dec. Making with GlS 


3.23 


18 


CASE & Syst Tools 


3.22 


19 


GUI Interface 


3.13 


20 


Visual Basic 


3.12 


21 


Spatial Analysis 


3.03 


22 


DSS/EIS 


3.01 


23 


Structured Programming 


2.98 


24 


Internet & Web Applications 


2.88 


25 


ArcView Software 


2.13 


26 


C Language 


1.95 



Systems life cycle is in third place, fifth, and sixth 
place interspersed between interpersonal 
communication and organizational knowledge. Data- 
base design and concepts are sixth and seventh. From 
the standpoint of curricular revision, this points to the 
need to put even greater emphasis on these behavioral, 
organizational, and ethical areas. These are long time 
mainstays of the systems field and model curricula. 
Systems develop-ment has proven to be a robust 
knowledge area over a number of iterations of national 
model curricula (DPMA, 1990, 1997). It is likely that 
systems development will be retained in the BSIS 
2000 curriculum, although it will take a different form 
i.e., it will likely be more oriented towards rapid 
development in the context of the web and 
dynamically changing environments. 

Database is the most valued technical curricular area 
for alumni career advancement. This again reflects an 
enduring mainstay feature of national model curricula. 
Databases have persisted in importance in IS. They 
remain so in the internet age, since database underlies 
many aspects of e-commerce and e-business. 



Alumni rate the career impact of ethics in the upper 
moderate range. It may be that BSIS graduates, with 
rising career attainment and often managerial or high 
level systems status, run into ethical issues to an 
increasing extent. 

At the lower end of perceived career benefits are a 
number of technical areas including C language, 
ArcView software, internet and web applications, and 
structured programming. By contrast, those technical 
areas were all in the middle range for current student's 
curricular knowledge gain. There are several 
explanations. First, some of these areas are so new 
(ArcView, internet) that most of the graduates could 
not have been exposed to them in the BSIS or outside 
of the university at the time of their BSIS schooling. 
The C language and structured programming may be 
less important because of the occupational rise of 
alumni towards management and senior systems 
positions that involve less hands-on programming. 
This is reflected in the respondent profiles which show 
that only eight percent of alumni are in programmer 
positions. At the same time, only two percent of the 
sample of current students are programmers. 

The lower perceived career value of technical 
curricular objectives underscores the fast movement of 
technology, so future BSIS alumni may not be able to 
utilize the technical skills they are learning today on a 
career basis. This is contrary to the sometimes 
strongly stated desire of current students to master 
technical skills. 

Generally, the alumni findings on specific curricular 
areas point to the importance of the "softer" behavioral 
elements in the curriculum as well as the long-time 
mainstays of systems development and database. 

The results by gender indicate that alumni male BSIS 
students have greater career impact of the program. 
The only statistically significant difference is for the 
career impact of communicating the project, with 
males having higher impact. This is unexplained. 

The results of the first open-ended question for current 
BSIS students provide valuable additional 
suggestions/comments on the curriculum. In 
particular, question I concerns improvements in the 
BSIS curriculum. The suggestions focus on (I) 
expanding and improving the technical content, (2) 
adding optional workshops, tutorials, and elective 
courses that might even include certificate training. 



202 Proceedings of the 1 Annual Conference of the International Academy for Information Management 



such as Microsoft SCCE. In the technical realm, 
students are particularly interested in expanding the 
Visual Basic coverage, perhaps by adding another 
course. 

The results of the second open-ended question concern 
the value of the non-major courses in the BSIS 
program. There seems to be diverse opinions about 
Mgmt. 310 (Philosophical Foundations of 
Management) ranging from strong support to 
dislike/request to eliminate it. Looking at the parts of 
Mgmt 310, the humanities segment seemed more 
supported than the portfolio part. There was also a 
suggestion to separate the course into two courses and 
move the portfolio part later in the curriculum. The 
student response on the Mthw 303 (Mathematical 
Applications for Information Systems) course was 
mixed. We can assume that this may depend on the 
extent of individual student preparation and particular 
instructors. Reflecting other responses, some students 
may need tutorials and workshops to enhance 
readiness to take Mthw 303. The responses on the 
Mgmt. 330, Buad 337, and Buad 469 courses were 
generally good. 

Another aspect of the assessment of student work in 
the BSIS Program is the student experience with 
senior project. An evaluation was conduced by Jeff 
Kim of a random sample consisting of BSIS senior 
projects, with 3 project reports in 1998; 3 in 1999; and 
4 projects from 2000. The findings indicated the 
following accomplishments in the senior project: (1) 
high level of systems implementation. (2) 
documentation report was visible, and (3) students 
followed the project report structure offered by the 
project guidelines. However, the evaluation pointed to 
the following areas of weakness or challenge. No 
analysis was done on the real world organizational 
context of the projects. This is important for several 
reasons. Studies are increasingly pointing to the 
, organizational aspect as important to systems success. 
Second, the organizational context touches based as a 
capstone element with the organizational/ managerial 
side of the BSIS curriculum. The BSIS 3 year goals in 
section 9 point to this area. Secon, no analysis was 
done in the sample projects on workflow, except in the 
case of “Use Case” analysis. Again, this is an 
important concept today for the practical success of 
systems, and needs to be emphasized in project 
enhancements in the future. Third, each stage of the 
project design and implementation appeared to be 
disjointed. In other words, the project calendar of 



“piece work” assignments that are put together at the 
end may be discouraging an integrated series of 
project tasks that are coordinated together. This can 
be improved by modifying the BSIS project guide to 
stress returning sometimes to earlier concepts and 
prior activities and steps, and to generally being more 
holistic. Lastly, the project reports did not reflect 
enough of what the students had learned from the 
exercise of doing a project, with the exception of one 
report written in 2000. This is a part of “reflection,” 
which correlates with missions and would add to the 
learning and experience of projects. 

The goals and objectives later point to areas for 
improvement of the senior project. The project 

assessment also points to the need to assess the timing 
and sequence of the project and project related (e.g. 
ISYS 404) courses; this can be considered with the 
present curriculum, prior to the BSIS 2000 
implementation. A final point is that the focus of 
knowledge and learning may need to be refocused. 
We need to think through the goal overall of the 
project. Is it to perform systems analysis, write up 
requirements, program, and evaluate the outcome? Is 
it keyed to preparation for jobs and hiring on 
graduation? Or is it to reflect on the principles of 
systems design? These need to be thought through 
more and articulated to students better, as well as 
practiced in the way decided upon. 

As an example of the assessment findings, the survey 
of current students received Likert scale responses on 
the extent of current knowledge of 26 program 
objectives. For N=53 responses, the average current 
knowledge was 2.66 on a 1 to 5 Likert scale, with 5 
being the highest value. This reflects an intermediate 
state of current knowledge, which is to be expected for 
an even distribution of students over the two years of 
the program. The comparison of students in the first 
half of the program i.e., junior year to those in the 
second half i.e., senior year shows an average current 
knowledge of juniors of 2.55 and of seniors of 2.79, 
for an average gain of 0.25, or about ten percent. 
When juniors are compared with students who were 
seniors in the last six months or had just completed the 
program, the average gain in current knowledge was 
0.42 or about sixteen percent. These results provide 
confirmation of the learning value of the IS major. We 
interpret that the gains are not higher because the 
student population consists of working adults students, 
most of whom come into the program as IS or 
technology professionals, so their starting knowledge 
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base is higher than for a traditional younger resident 
student. 

The gains in current knowledge from the program can 
be further desegregated by the sixteen program 
objectives. Comparing juniors to seniors in the last six 
months or recent graduates, the following are the 
highest gains by program objective. 



1 


GIS 


1.36 


2 


Communication of a 
major project 


1.12 


3 


Relational Data-base 


1.12 


4 


Systems Dev. Life Cycle 


0.95 


•5 


CASE tools 


0.94 


6 


Data-base Applications 


0.92 


7 


Verbal Communications 


0.58 



Doing the same comparison, the areas of lowest gains 
(or losses) were the following. 

The information in the disaggregated table is 
invaluable in determining where the most learning is 
taking place and where learning is not as effective, so 
future improvements can be focused in those areas. 
For instance, the results confirm extensive learning in 
GIS, a new technology area for most students, in 
verbal communications, and in the mainstay areas of 
data-bases and systems development. Those are areas 
in the present curriculum that appear to be working 
well and will likely be retained in the new curriculum 
although modernized. 

On the other hand, areas that are not doing well as 
measured by where learning is taking place are the 
world wide web, networks, organizational systems, 
certain aspects of computer ethics, and DSS/EIS. 
These are pointing to areas in the new curriculum to 
seek major teaching and learning improvements. For 
instance, the university has been behind schedule on 
implementing its high speed campus network, which 
until recently has reduced the potential for learning of 
world wide web skills. Given the importance of this 
area in industry, it is something that is urgently 
pointed to for the new curriculum. The low score for 
networking may stem from the lack, until recently, of a 
hands-on networking laboratory for training on NT 
administration and configuration. Also, the 
networking course was in need of revision, a task 
which has recently been accomplished, but may need 
even more revision and attention in the new 
curriculum. 



Rank 



Program Objective 



Average gain in 
current 

knowledge from 
junior year to 
last 6 mos of 
curriculum plus 
recent grads 



Rank 


Program Objective 


Average gain in 
current knowledge 
from junior year to 
last 6 mos of 
curriculum plus 

recent grads 


21 


Dec Supp Systems/ 
Exec Inf Systems 


0.13 


22 


Understanding the 
Implications of 

technology 
advances for ethical 
issues 


0.12 


23 


Understanding the 
theoretical 
perspective of 

ethical philosophies 
as they relate to 
businesses and 

organizations 


0.10 


24 


Organizational 

Systems 


-0.12 


25 


Networks 


-0.24 


26 


World Wide Web 


-0.62 



The data from the alumni survey were equally 
revealing. For instance, on the question of how well 
the IS degree had impacted the alumni's career 
advancement, the following were the ten highest 
program objectives (on a 1 to 5 Likert scale, with 5 
being the highest impact). 



Rank 


Program Objective 


Rating of 

Impact of 

Program 
Objective on 
Alumni’s 
Career 

Advancement 


1 


Verbal Communications 


4.04 
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2 


Communication of a 
Major Project 


4.04 


3 


Organizational Systems 


3.96 


4 


Systems of Life Cycle 


3.96 


5 


Senior Project 


3.93 


6 


Relational Data-Base 


3.89 


7 


Be Able to Design and 
Build a Business 

Information System 


3.88 


8 


Data-Base Applications 


3.84 


9 


Understanding the 

Implications of 

Technology Advances for 
Ethical Issues 


3.72 


10 


Understanding Ethical 

Delemmas 


3.72 



The alumni results point to the importance of the softer 
sides of the curriculum on career impact — areas such 
as communications, organizations, conducting a 
project, and ethical issues. Systems life cycle is in 
between in "softness" and is ranked a high fourth. The 
more technical areas of data-base are present but 
ranked sixth and eighth. This confirms some other 
literature that have pointed to the "softer" areas as 
ultimately having more impact on IS graduates 
careers. From the standpoint of curricular revision, 
this points to the need to put even greater emphasis on 
these behavioral, organizational, and ethical areas. 

The results from the focus group sessions with 
students and the discussions with the corporate 
advisory committee were equally informative, but are 
not covered in this extended abstract, but will be in the 
full paper. Suffice it to say that they delved into 
certain issues in even more depth and provide even 
greater knowledge of current status of curriculum and 
the recommendations for the new one. 

In summary, this paper presents a new model for IS 
program assessment, with an emphasis on curriculum. 
Many stakeholders can be involved in a more 
extensive data-gathering process that provides more 
robust information on student learning, alumni career 
benefits from their education, and corporate advice for 
program and curricular improvement and change. The 
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assessment process serves to set a baseline than can be 
re-tested longitudinally over time to benchmark longer 
term program advances. The methods can be adapted 
for a wide variety of IS undergraduate and graduate 
programs. In another paper we suggest more detailed 
steps that programs can take to instill a full program 
and curricular assessment process into their strategic 
planning and development. 
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ABSTRACT 

The prior knowledge that a student brings into the lecture is one of the major factors influencing teaching 
effectiveness. It is therefore important that lecturers are able to ascertain the level of prior knowledge and adjust 
their teaching a ccording ly. This paper adopts a n iterative lea rning model that seeks to enhance teaching effectiveness 
by developing and leveraging prior knowledge and illustrates this approach in the context of 1ST teaching. 



The most im portant sing le factor influe ncing learn ing is what the learner already knows. Ascertain this and 
teach accordingly (Ausubel et al., 1978: 163). 



INTRODUCTION 

The aim of teaching quite simply is “to make learning 
possible” (Ramsden, 1992). While much of university 
education is based on the theory that students will learn 
if information is transmitted during lectures or if they do 
things in practicals or seminars, teaching as making 
learning possible represents a speculative and reflexive 
activity. This approach to teaching views the students, 
teaching and the subject to be learned as interactive 
components of an integrated model. From this 
perspective, teaching is an iterative process of 



continuous improvement based on the interactions 
between the framework components. 

One of the main attributes of the studentcomponent that 
influences the learning process is their priorknowledge 
of material relevantto the subject. A number of research 
studies indicate that the variance in students' prior 
knowledge is one of the strongest factors influencing 
their educational achievement and their understanding of 
the lecture material (e.g. Beckwith, 1991 ; Hadwin et al, 
1999; Yates & Chandler, 1991). Laurillard (1993) 
further argues that “it is impossible for teaching to 
succeed if it does not address the current forms of 
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studentunderstanding of a subject”. Prior knowledge can 
have positive and/or negative effects on learning. Pre- 
existing knowledge has a cumulative impact on 
individual developmenttherebyacceleratingthe learning 
process. However, existing knowledge can inhibit 
learningif it containsmisconceptions (i.e. faulty beliefs 
or knowledge based on misinformation). In recognition 
of this twofold impact of prior knowledge, the learning 
process should be directed so that it 1 ) builds on positive 
and consistent prior knowledge and 2) eliminates or 
reduces the impact of misconceptions. 

One of the difficulties in applying a learning strategy 
targeted at the level of students’ prior knowledge is the 
lack of information regarding these levels. Normally, 
lecturers receive feedback on students knowledge and 
understanding through formal assessment at the end of 
a teaching block. While such post-teaching assessments 
provide useful indicators of student performance they 
have a limited capacity for corrective effect on the 
teaching process. Ausubel atal (1978) emphasises the 
importance of checking on the prior knowledge - what 
students bring into a course - and using this to inform 
teaching. Where the lecturer is unable to accurately 
ascertain the level of prior knowledge relevant to course 
content, it becomes difficult for students to su ccessfully 
participate in exercises that require the application of 
prior knowledge. Furthermore, since lecturers will often 
address groups with different levels of prior knowledge 
this can cause problems in determining the level at 
which teaching should be targeted (Entwistle, 1998). In 
an effort to ensure that students have a common 
understanding of concepts, the lecturer may then find 
him/herself having to spend valuable lecture time 
conveying large amounts of information, rather than 
focusing on higher level goals of analysis and synthesis 
(Jenkins, 1994). It is therefore important to not only 
determine existing knowledge but to also ensure that 
there is an existing level of shared awareness of required 
concepts. 

Finally, Yates and C handler (1 991) argue that possessing 
knowledge is not equivalent to using this knowledge for 
achieving cognitive, learning or mnemonic goals. As 
knowledge is inert, failure to activateprior knowledge in 
a problem solving environment obstructs the learning 
process. Willoughby et al. (1993) employ an 
interrogation technique to activate existing prior 
knowledge and boost learning. Christen and Murphy 
(1991) argue that challenging thestudentsto call ontheir 
prior knowledge transcends the learning process from 
memorisation to meaningful learning. 
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Ramsden (1992) suggests that excellence in teaching 
requires constant attention to how a subject is being 
understood by students, and the ability to use the 
assessment to change instruction so that it more 
accurately addresses student's errors and misconceptions. 
This paper is therefore premised upon the need to more 
accurately understand what students know about a 
subject (prior knowledge), to enrich their prior 
knowledge and to use this understanding to inform 
teaching. The remainder of this paper introduces, in the 
context of a case study, an iterative learning model that 
incorporates this process of enrichment, evaluation and 
activation of prior knowledge on a particular subject or 
learning unit. This model has been formulated on the 
basis of a review of literature on prior knowledge and 
adopted in the context of 1ST teaching. 

THE CASE STUDY 

Information Systems and Technology is a year-long 
undergraduate Level- 100 course with a current 
enrollment of 447 students. There are no pre-requisites 
(or restrictions) for entry into this course. The student 
population consists of, a cross-section of students: 
intending majors in Computer Science (CS) or 
Information Systems (IS); and other-majors (eg. 
accounting) who want to be IS-literate but not IS- 
professionals. Some students will take this course 
having taken or concurrently taking Level-100 courses 
in CS. Others may be Level-200 ofLevel-300 IS or CS- 
majors needing the course to complete the credit 
requirem ents for their degree. These characteristics lead 
to significant differences in the prior knowledge of the 
student population. 

The course aims to develop, in students, an 
understanding of Information Systems and Technology 
and its applications to modern business needs including 
the application of information systems models to the 
analysis of business situations. ‘ A large portion of the 
lectures are dedicated to the application of knowledge 
using active participation techniques. It isthe experience 
of the lecturers in this course that the success of active 
participation within a large class setting is impacted by 
the aggregate level of content knowledge in the 
particular subject area. If the aggregate level of content 
knowledge can be ascertained, then the lecturer is able to 
balance materials delivery and applied learning. In 
addition, the level and content of the applied examples 
can be targeted at the appropriate level. 
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To evaluate prior knowledge, students are required to 
participate in weekly on-line tests. These tests are 
accessed via the WWW and are normally available fora 
week prior to the respective lecture. Students are given 
10 minutes in which to complete five (5) randomly 
selectedmulti-choice questions. Submissions are marked 
on-line and feedback is immediate. Students are awarded 
1/2% point for each test; the final mark is based on the 
best 10 submissions (maximum of 5% in total). Since the 
final mark is determined using the best 10 of the test 
sittings, students are thereby encouraged to participate 
without fear of failure. Approximately 60% of the 
students complete the weekly on-line tests. Prior to the 
lecture session, the summ arised test results are reviewed 
by the lecturer to distinguish those topics on which 
students performed well or poorly. The distribution of 
incorrect responses for each question is also reviewed. 
The following section discusses, in the context of the 
above case description, the adoption of a four-stage 
iterativelearning model that seeks to determine {enrich). 



evaluate, and leverage {reflect and activate) prior 
knowledge. 

THE ITERATIVE LEARNING MODEL 

To improve the effectiveness of the teaching process, a 
four stage evolutionary model of learning is proposed 
that builds on and utilises students' prior knowledge of 
relevant material (Figure 1). The model supports the 
concepts of determination and activation of prior 
knowledge. In the first two stages of the model, prior 
knowledge is built and assessed. The third and fourth 
stages demonstrate how teaching can be adjusted to 
leverage and activate prior knowledge. 

In the first stage {Enrich) students are provided with 
recommended readings for the forthcoming lecture. This 
encourages students to develop a common knowledge 
base prior to the lecture. The readings normally consist 
of both theoretical definitions and facts (declarative 
knowledge) andproblem scenarios(appliedknowledge). 



FIGURE 1 

THE ITERATIVE LEARNING MODEL 



During the lecture 

◄ 



Prior to lecture 






1 advance to enrich and ensure 




Activate 


1 shared prior knowledge 

1 


Evaluate 



Stage 4: Challenge 
students to apply prior 
knowledge in context 




Stage 2: Assess the level 
of prior knowledge for 
individual students 



Stage 3: The lecturer learns from 
the aggregate knowledge level and 
the individual misconceptions to 
plan next lecture 
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To encourage preparation for lectures and to help 
determine their understanding of such material, in the 
second stage {Evaluate) students are required to take a 
weekly on-line (WebCT-based) test While ' the 
technique of on-line testing is not new, these tests focus 
not on an assessment of students' understanding of 
taught material, but on determining the students' prior 
understanding of forthcoming material and using this to 
inform teaching. An analysis ofthe responses contribute 
to a better understanding of students' prior knowledge, 
and their errors and misconceptions. The process of 
evaluation also provides an incentive (by way of credit) 
that encourages students to prepare for lectures 
(enrichment) and participate in on-line testing. 

In the third stage {Reflect), the lecturer's understanding 
of students' prior knowledge is used to inform teaching 
practice in the lecture sessions (Ausubel et al, 1978). 
While assessment often provides feedback to students on 
how they should learn (or have failed to learn), this form 
of assessment is designed to provide feedback to 
teachers on how to teach. The lecturer is able to tailor the 
session to appropriately balance content delivery, 
clarification of misconceptions, and analysis and 
applicationof concepts. In the context ofthe case study, 
the summarised results from the on-line tests are 
reviewed and appropriate annotations and emphases are 
made to the prepared lecture. The lecturer is then able to 
place additional emphasis on the material that students 
did not grasp as well as correct misconceptions in prior 
knowledge. 

The final stage {Activate) of the Iterative Learning 
Model involves the activation of prior knowledge. 
Duringthe lecture, students are challenged to recall prior 
knowledge as they apply itto problem-solvingscenarios 
and link new concepts to pre-existing ones. Activation of 
knowledge allows the students to appreciate “how and 
when existing mental elements can bear upon new 
demands” (Yates and Chandler, 1991). The declarative 
knowledge accumulated through prior reading is 
transformed into procedural knowledge that is bound by 
context (e.g. through case-based scenarios). This further 
enriches student knowledge in the subject area and 
creates a progressive accumulation of knowledge. 
Hence, learning becomes an iterative process. 

CONCLUSION 

The knowledge that students bring to the lecture is one 
ofthe most important factors influencing their learning. 
Since effective teaching is that which makes learning 



possible it is of great importance that lecturers 
understandthe level of the students' priorknowledge and 
target their teaching accordingly. This paper has 
proposed an iterative learning model that aims to 
improve teaching effectiveness by building and 
leveraging the prior knowledge ofthe learner. The model 
has been adopted in the context of teaching and learning 
in IS education. Based on the iterative learning model, 
this paper describes a technique (through on-line testing) 
that can help determine and leverage the prior 
knowledge of students, for informing teaching. Future 
research could undertake an analysis of the impact of 
this approach on teaching effectiveness as measured by 
student understanding, motivation and performance.^ 

END NOTES 

1. Source: Department of Accountancy, Finance and 
Information Systems: Undergraduate & Graduate 
Courses 2000, University of Canterbury, NZ, p. 12. 

2. It is expected that preliminary results of the impact of 
on-line testing on teaching effectiveness and student 
learning will be available at the end of the 2000 
academic year. 
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ABSTRACT 

The IT/IS education sector needs to come up with creative ways of thinking about on-line education. In this paper, 
the major themes in the literature on on-line education to date are highlighted with a view to identifying issues that 
are either missing or under-emphasised. Next, the “radical model of on-line teaching” is presented. The paper is 
concluded with a discussion of how the model addresses some of the issues that are missing or under- empha sised in 
the literature. 



INTRODUCTION 

On-line education can be defined as teaching and 
learningactivitiesenable dby electronic media. Given its 
relianceon emerging technologies, it is a relatively new 
area of research and practice for IT/IS educators. At the 
same time, the unique expertise that IT/IS educators 
have and the crucial importance of this area for the 
higher education sector in general, creates a situation 
where in many universities IT/IS academics are called 
upon to lead the transitionof their universities to on-line 
education. 

As part of this new role, it is also IT/IS educators who 
are expected to champion on-line teaching by being the 
first to use it in their own teaching. It is expected that 
based on their personal experience with this teaching 
mode, they will be able to disseminate knowledge about 
it to their less computer literate colleagues. It is also 
expected that based on their experiences as early 
adaptors and champions of this practice, they would be 
able to teach the rest of the academic sector how to do 
on-line teaching painlessly and more effectively. 



It is for this reason that the IT/IS education sector needs 
to come up with creative ways of thinking about on-line 
education. Doing so will not only improve our own 
practices but help us enlighten others within the higher 
education sector on how to avoid our mistakes, 

WHAT ARE THE MAJOR EMPHASES IN THE 
CURRENT LITERATURE ON ON-LINE 
TEACHING AND LEARNING? 

The literature on on-line education to date seems to 
emphasise a number of themes: 

The Advantages and Disadvantages of Teaching On- line 

A major theme in the literature on on-line education is 
the advantages and disadvantages of this practice. Cost 
effectiveness and flexibility, particularly forstudents, are 
often mentioned as the major advantages of on-line 
teaching (West, 1998; Cunningham, 1998). While, 
"second rate" education and extra work for the lecturer 
are mentioned as the major disadvantage (Cunningham, 
1998). There is general agreement in the literature, 
however, that no matter what the disadvantages are, the 
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cost-effectiveness of on-line education is so compelling 
that it is bound to be the way of the future for 
universities (Cunningham, 1998; Ryan, 1998; Flew, 
1998; Thomas, Meredyth, and Blackwood, 1998 and 
others). 

The Range of Technologies for On-line Teaching 

Another important theme in the literature on on-line 
education is a discussion of the ways in which it can be 
accomplished. In particular, the publications in this area 
contain descriptions of teaching situations (quite often 
borrowed from the face-to-fece mode) that lend 
themselves to the on-line mode. For example, lecture 
notes can be placed on the class web site (just like 
distribution of these notes to the students in the face-to- 
face mode), students can submit assignments and be 
tested on-line, on-line multimedia packages can be 
developed to guide students through the major 
comp onents of a subject area (e.g., teach students how to 
use a software). There is also emergingliterature on how 
video and audio conferencing can support teaching and 
learning and some literature on how chat groups can be 
used to support teaching or as the basis for class 
interaction (West, 1998; Ryan, 1998; Cunningham, 
Tapsall, Ryan, Stedman, Bagdon, andFlew, 1998; Kelly 
and Shing Ha, 1998; Tsang and Fong, 1998). 

The Profound Changes to the Role of the Lecturer as 
a Result of On-line Teaching 

Itis implied in the literature that on-line teachingheralds 
a new role for academics, with a stronger emphasis on 
the lecturer as provider of mass education and a lesser 
emph asis on his/her role as creator of new knowledge or 
as a researche r. This theme o ften leads to the conclusion 
that in the future academics will be expected to engage 
in less research and more teaching. 

WHAT IS MISSING IN THE 

CURRENT LITERATURE? 

A careful reading of the literature in this area seems to 
suggest that several themes are either missing or under 
emphasised in this literature. These themes include: 

A Relative U nder-em phasis on the ’’Many to Many” 
Mode of Student/lecturer Interaction 

The discussion of on-line learning seems to suggest three 
modes of interaction between lecturer and students 
which are mutually exclusive: 



’’One to one”. When students are reading materials on 
the class web site, getting assessed on line or working 
with an on-line tutoring package, they are basically 
interacting with the material on a one to one basis. 

’’One to many” . When students are in an audio or video 
conferencing situation, as well as when they 
communicate directly with the lecturer through the 
telephone, or perhaps when they read materials that are 
updated occasionally by the lecturer on the class web 
site, they can be said to interact in a "many to one" or 
"one to many" mode. 

’’Many to many”. When students are involved in 
discussion with other students as part of theclass regular 
interaction or as an activity that is supporting the 
learningprocess, it would be an example of the "many to 
many" mode of interaction. 

In contrast to the first two modes of on-line interaction, 
the "one to one" and "one to many", that seem to be 
predominant in the currentliterature on teaching on-line, 
the third mode, the "many to many", is not as frequently 
practiced as part on-line teaching. Even when this mode 
is practiced, it is more than likely to be seen as a parallel 
process to the real teaching, intended to "keep the 
students motivated". Thus, when the "many to many" 
mode is practiced it is not used as a teaching activity on 
its own (where stude nts present the ir work, comment on 
other people's work, and get their work assessed on line) 
but rather as an enhancement of the real teaching, which 
takes place in the "one to one" or "one to many" mode. 

An Under-emphasis on the Soft Knowledge/Skills 

A study (Lee, Trauth and Farwell, 1995) based on 
combined input from industry and academia, concluded 
that four major clusters of knowledge/skills will be 
required of IS personnel in the next few years: 

1) Technical Specialties Knowledge/Skills: including: 
operating systems, programming languages, 
database management systems, networks, 
telecommunications, etc. 

2) Technology Management Knowledge/Skills: 
including issues such as where and how to deploy 
information technolog ies effectively and profitably 
for meeting strategic business objectives. 

3) Business Functional Knowledge/SkiWs: including 
how to re-engineer business processes before the 



Proceedings of the 15*’ Annual Conference of the International Academy for Information Management 



211 



adoption of a new information system to produce 
maximum benefit from the system. 

4) Interpersonal and Management Knowledge/Skills'. 
which relates to the “boundary-spanning” role of 
information systems personnel. This role requires IS 
professionals to master interpersonal skills such as 
“selling,” "negotiating,” "leading,” and "coun- 
selling.” 

It is worth noting that three ofthe four knowledge/skills 
clusters identified by Lee et al, namely, technology 
management, business knowledge, and interpersonal 
skills are not the “hard skills” traditionally associated 
with IT/IS education. These three clusters can be 
described as “soft” because they emphasise an 
understanding and ability to work with people rather 
than machines. The authors also report that while 
business managers are reasonably satisfied with the 
skills that IS graduates have in the "hard" areas, they are 
dissatisfied with the students’ knowledge and ability to 
apply the soft skills. They conclude that it is the soft 
skills that need more attention from IS educators. 

Given the importance of soft skills in IS education, the 
question arises how can they b e most e ffecti vely tau ght? 
Given that these skills are related to interpersonal 
interaction, it would seem logical that the best way to 
teach them if not face-to-face, would be through the 
many to many mode. Obviously, the overemphasis on 
the one to one and one to many mode of interaction in 
the on-line teaching practice results in an over-emphasis 
on hard skills and under-emphasis on the soft skills 
needed by IS/IT professionals. 

An Under-emphasis on Flexibility for the Lecturer 

Much of the current discussion about on-line learning 
and teaching emphasises the flexibility that results from 
this mode of learning to students. Students are supposed 
to be able to learn any time, and anywhere (as long as 
they have access to the Intenet). They are supposed to be 
able to not purchase books (if the course materials are 
available for them on the class Web site) or other 
teaching materials. Given the cheaper price of on-line 
education, it is supposed to be available to all students, 
irrespective of income, socio-economic level, gender, 
family status, or professional activities. In fact, the 
general assumption is that on-line education would make 
it possible for full time employees to do their studies 
“flexibly” in between all their other daily activities 
without major investment in either time or energy. 



In contrast to the above, there is almost a universal 
agreement in the literature, that on-line teaching results 
in less flexibility and more work for lecturers. In fact it 
would be true to say that the increased flexibility for the 
learner is seen as associated with a decrease in flexibility 
for the lecturer. It is precisely for this reason thaton-line 
teaching, as a major new activity for lecturers, is 
associated with the belief that academics in future w ill 
be disseminators of knowledge (teachers) rather than 
creators of knowledge (researchers). 

WHY DO WE NEED TO BE 
CREATIVE ABOUT ON-LINE EDUCATION? 

If we are to think creatively about on-line teaching and 
learning, we would have to address the three issues, 
outlined above, that are currently missing from much of 
the thinking in this area. In other words, our future on- 
line teaching models should incorporate ALL modes of 
interaction (including the "many to many"), ALL 
knowledge/skills clusters (including the soft skills) and 
the interests of both lecturers and students. If this new 
education practice is to succeed, it would have to be cost 
effective to both lecturers and students.lt will also have 
to be inherently motivating to both students and 
lecturers, and organisationally viable. 

There are obviously different ways to reach this goal. In 
the following sections an approach to on-line education 
that is currently p racticed by some fa culty in the F acuity 
of Informatics and Communication is presented. This 
model has been used with a range of courses, including 
small post-gradu ate courses (with up to 20 students) and 
large under-graduate courses (with up to 100 students). 
The students are a combination of on-campus and distant 
learners. Both groups are treated as one homogenous 
group, which means that the course does not have any 
face-to-face teaching. 

To date, this model has been used to teach courses in 
Management of Information Systems and Electronic 
Commerce. Student responses to this model have been 
so positive that even though the thinking about it is still 
evolving, the authors believe that it is ready to be shared 
with others. 

The "Radical" On-line Teaching and Learning 
Model 

The teaching materials forthe Radical Model of On-line 
Teaching (irrespective of what area is being taught) 
include: 
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• a video which contains detailed explanations from 
me on how the course is run; 

• a ten-page booklet "Course Outline" which describes 
all necessary information about the course(it is 
available on line as partof the course’s Web site and 
is provided to the students on a CD ROM and on 
hard copy); 

• a textbook; and 

• a class e-mail list. 

The first thing that students are expected to do once they 
read the Course Outline an d watch th e video is subscribe 
to the class e-mail list. They are then expected to 
introduce themselves to the class on-line so they can be 
divided into weekly presentation groups. The allocation 
to groups is completed by the second week of the 
semester. By this tim e, students are expected to establish 
contact with their virtual group members and start 
working on their assessment tasks. On week 3 of the 
semester, the first group makes its presentation to the 
class on-line. The presentation consists of an article 
(which the students have to enclose, attach, or simply 
establish a hiper-link to) and a critique that links the 
article with the reading in the book for the week. 

The presentation is expected to be made on Tuesday of 
each week. By Friday, each of the groups in the class is 
supposed to comment on the presentation. On Sunday, 
the presentations for the week are read by the instructor 
along with the comments that were made by all the 
groups. All groups are marked every week for either 
their presentation or their comments about other 
students* presentations. This procedure is repeated for 
ten weeks u ntil the end of the semester, with each week 
dedicated to an in-depth discussion on a different topic 
that is related to the reading for that week. 

What Are the Advantages of this Approach and Why 
It Should Be Considered Radical 

This approach encompasses the best of all three 
modes of student/teacher interaction. Students have 
some interaction with the material (when they read the 
book and the articles that are presented by the other 
groups). They have some interaction with the lecturer, 
through direct correspondence on e-mail and the replies 
to them about their presentations on the class list. 
However, the major bulk of their interactions is in the 
"many to many" mode, with the other students in their 



presentation groups and with the rest of the students in 
the class through the class e-mail list. 

Throughout the semester, students are assessed on 1 1 
assessment tasks (including their group presentation, 
comm ents on other students’ presentations and an end of 
term exam). For each presentation that they make, they 
get (if the class consists of 100 students) 19 comments 
which represent the views of their own group members 
(in this case, there will be 10 members per group) as well 
as all other 90 students in the class. Since this procedure 
is repeated every week, the students receive over 100 
inputs from their group members, the other groups, and 
from the lecturer by the end of the semester. 

It should be noted that even though class interaction is 
the means through which teaching takes place, the 
radical model does not resultin the listbeing flooded by 
e-mail messages. Students are instructed to refrain from 
using the class list for unlimited expression. The place 
for such interaction is supposed to be the small 
presentationgroups which they establish to support their 
group work. The messages that end up being posted on 
the class list are messages from the list moderator (the 
lecturer) "formal" presentations of the students’ work, 
and comments by the other groups about these 
presentations. 

This approach encompasses both hard and soft 
knowledge/skills. In addition to learning about the 
content area for the semester, students learn important 
on-line skills such as how to set up their e-mail lists, 
how to be citizens of an on-line community, and how to 
contribute to a virtual team, including dividing the work 
between the team members, resolving conflicts, 
developing ideas and projects, and providing positive 
feedback to others about their work. 

Through the involvement of students from diverse 
backgrounds (many of whom are fully employed) 
students learn about how organizations use the abstract 
concepts that are mentioned in the readings. They also 
learn about relevant legislation and ethical issues. 

This approach is flexible for both lecturer and 
student. The flexibility to the students, which has been 
discussed before, is increased in this approach because 
the students don't have to submit hard copy assignments 
(hence, nothing can get lost through the system). They 
get to know if their submission was successful 
immediately when they see it posted on the class list. As 
well as this, if something happens to preclude an 
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individual student’s contribution during the semester, he 
or she can negotiate with their group members on how 
they can take time off and compensate later by doing 
more work for the group. In fact, students don't need to 
ever negotiate with the lecturer on late submission, 
special consideration, etc. All negotiations on these 
issues are carried out within the group. 

Students have further flexibility in not having to down- 
load large amounts of data from the class Web site (there 
is nothingon the web site other than the Course Outline). 
They don’t need to buy any books other than the course 
textbook, and even this book can be shared between 
them up until the end of the semester, as all assessment 
tasks are group based. Because all learning is facilitated 
by the class list, the students can engage in class 
activities from home, work, or while travelling. Further 
flexibility to the students is provided through the 
students’ selection of supplementary readings for class 
discussion by themselves. As a result, students get to 
read quite a large number of articles on topical issues 
that are of interest to them rather than forced to read 
articles selected by the instructor. 

Lecturer flexibility is also an enormous benefit from the 
radical model. Since thepackage forthis course doesnot 
include a Study Guide, there is no need to update one 
every semester. Since the course is in noway dependent 
on a textbook, there is no need to modify or change it in 
any way if and when there is a need to change a 
textbook. In fact, preparing study materials for a new 
semester should not take more than a few minutes, given 
that nothing substantial has to change. 

As for on-going teaching; reading the weekly 
presentation and the comments by the other groups 
(students are restricted to two pages or two screens 
maximum per critique or comment on other people’s 
critique), takes about an hour and a half to two hours per 
week. This can be done from anywhere, including from 
home or from a conference. Theoretically, even if the 
lecturer is totally incapacitated, anotherperson can easily 
take over and do the on-going weekly assessment, 
without inconveniencing the students. 

Note that this design is also advantageous from a legal 
perspective. Since articles by other authors are not used 
as part of the course Web site, there is no infringement 
on other people’s copy -rights. 

The most important aspect about this model is that no 
matter how m any stude nts are in the class, the amount of 



work for the lecturer is the same. No matter how many 
students are in the class, 10 or 100, the lecturer ends up 
checking 10 presentationsof one page eachper week for 
ten weeks. If the class consists of 10 students, these 10 
pages of text represent the work of each of them. If the 
class consists of 100 students, the ten pages will 
represent the work of the ten groups into which the 
students have been divided. Thus, the amount of 
marking for the lecturer remains the same, irrespective 
of the number of students in the class. 

This approach can be enhanced by other means, of 
course; 

1 ) class interaction can be sup ported by audio or video- 
conferencing (In this case differences in time-zones 
between different locations where students were 
located around the world prevented this application); 

2) submissions can be made via the telephone (for 
example, students can tape role plays in a class on 
interpersonal communication and submit these to a 
voice mail box where they can be listened to and 
commented on by the other groups in the class); 

3) class assessment can be complemented by on-line 
testing (this has not introduced so far in this 
situation); 

4) teaching can be enhanced by on-linetutoring; andat 
least part of the interaction with the students can be 
undertaken through the class Web site, rather than 
on the class list. 

Other improvements to this approach could involve a 
class management system which would allocate students 
to groups and establish gro up lists for them . Currently in 
the absence of such a system, the allocation of students 
to groups is conducted manually and students establish 
their groups on their own. It would seem that a class 
management system and/or the use of team teaching 
would be necessary once the number of students in a 
virtual class where this model is applied goes beyond 
100. If this were to happen, the model could be easily 
duplicated, with more than one lecturer fecilitating the 
learning process. 

However, even without these enhancements, important 
lessons from use of the Radical Model are that this 
approach does not result in more work for the lecturer or 
in inferioreducation for the students. Student comments 
are very enthusiastic and they consider this format an 
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improvement over the usual hard copy distant education 
packages. They also consider it superior to m any of their 
face-to face-courses, where the ’’one to many" or "one to 
one" modes are practised. 
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A REVIEW AND ASSESSMENT OF 
TEACHING COBOL IN AN IS PROGRAM 



Diane Fischer 
Dowling College 



ABSTRACT 

Academic programs undergo periodic review and assessment. Reasons include requirements of accrediting 
agencies and a need to maintain competitive advantage in attracting students. In addition, IS programs face a 
rapidly changing environment. One area of interest in IS programs is the set of programming languages offered. 
These languages should be in step with the IS environment, other computer programs at the institution and the 
student audience. In particular, the offering of Cobol as a required language must be reviewed carefully. 

This study of an IS program *s Cobol requirement at a small private liberal arts college examines the background 
described above. The IS environment is investigated through a survey of job advertisements in a selection of on- 
line newspapers and employment services. In addition, a brief survey of language offerings in over 90 IS 
programs throughout the world is presented. An informal review of technical book offerings at a mega bookstore 
is also presented. Finally, the student body is described and an assessment of Cobol is given. 



INTRODUCTION 

Academic programs offered by a college or university 
undergo periodic review and assessment. There are 
many reasons for initiating this process. A major one 
is that many institutions seek to maintain accreditation 
granted by regional and state agencies. These agencies 
conduct periodic reviews of institutions and their 
programs. In particular, regional accrediting agencies 
in the United States - such as Middle States - are 
placing an increasing emphasis on institutions to 
review their programs, assess the quality and use this 
review and assessment to modify their programs in 
light of their mission. This process has been labeled 
“outcomes assessment.” 

A second reason for carrying out a periodic review and 
assessment is that in the United States institutions face 
an increasingly competitive academic market. In any 
region there are many institutions at all levels - from 
Junior colleges to universities. Most offer a similar 
array of programs to what has been a shrinking 
number of prospective freshmen. The growth of 



distance learning programs provides additional 
competition and removes the traditional geographic 
boundaries. Institutions must periodically review and 
assess their programs to maintain currency and 
attractiveness. 

These reasons for program review and assessment - 
the requirement of accrediting agencies and a need to 
attract students - apply in general to any programs. A 
third reason - a rapidly changing environment - 
compels individual programs. In particular, the IS 
environment is changing rapidly and has been doing so 
since its birth in the middle part of the 1900s. In order 
to continue to offer courses that provide an 
understanding of and tools for this changing 
environment, IS programs must undergo periodic 
review and assessment. 

One area of interest is the set of programming 
languages an IS program offers. There have been 
many programming languages. Some, like APL, have 
had a short shelf-life while others, like Fortran and 
Cobol, seem to live forever. The particular languages 
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an IS program offers should fit with the outside IS 
environment, computer offerings of other department 
at the institution, and the type of student the program 
attracts. 

Cobol has long been a mainstay of IS programs. But 
newer, more popular languages compete for a limited 
number of course offerings at many institutions. It is 
thus important to ask the question: “Should we 
continue to require that IS students take Cobol?” 

THIS STUDY 

The author selected the following languages to 
examine: C++, Visual Basic, Cobol, Java and Html. 
While the last is not strictly a programming language, 
it is offered by many programs. 

The IS Environment: The Job Marketplace 

In the past surveys have used classified ads to identify 
IS skills in demand. An Annual Skills Survey 
undertaken by ComputerWorld (ComputerWorld, 
1998), found the following % of companies hiring for 
these skills for 1999: 



Visual 

Cobol Basic C++ Java Html 

1 7% 1 6% 1 3 .25% 13% 1 4% 

A recent survey (Papp, 1999) that focussed on the 
New England area found the % of ads mentioning 
these languages as follows: 

Visual 

Cobol Basic C++ Java Html 

16% 10.75% 13.25% 5% 3% 

For this study, a brief survey was undertaken of jobs 
available through on-line newspapers and employment 
services. The newspaper selection included: The 
Washington Post, The LA Times, The New York 
Times, and The Atlanta Journal-Constitution. 
Employment services included hotjobs.com and 
monster.com. The jobs were compiled on March 20, 
2000. Results are presented in Table 1. 



TABLE 1 



Source 


City 


Cobol 


Visual 

Basic 


C++ 


Java 


Html 
















Hotjobs.com 


DC* 


2 


48 


107 


126 


121 




LA* 


31 


153 


11 


4 


13 




NYC* 


81 


341 


9 


7 


9 




Dallas* 


18 


82 


245 


237 


280 




Atlanta* 


17 


101 


256 


265 


233 
















Monster.com 


DC* 


19 


173 


608 


448 


1000++ 




LA 


113 


498 


1000++ 


1000++ 


1000++ 




NYC 


80 


384 


1000++ 


1000++ 


1000++ 




Dallas 


68 


450 


1000++ 


1000++ 


1000++ 




Atlanta 


21 


79 


290 


1000++ 


1000++ 
















W ash i ngtonpost .com 


DC* 


195 


955 


5616 


1857 


1780 
















Latimes.ocm 


LA* 


6 


36 


47 


44 


38 
















Nytimes.com 


NYC* 


12 


33 


836 


106 


73 
















Ajc.com 


Atlanta* 


4 


15 


31 


33 


22 
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Notes: * indicates metro area. 1000++ 
indicates maximum reported by monster.com. 
Many of the above sources list the most jobs for C++, 
Java and Html. Hotjobs.com indicates a top market 
for Visual Basic in the DC, LA and NYC metro areas. 
Note that Cobol jobs are still available, though other 
languages in most cities dwarf its numbers. 

These studies show the demand for Cobol jobs and 
skills have decreased. While regional differences may 
account for some of this decrease, the Y2K problem 
has come and gone and presumably removed a large 
source of the need for Cobol programmers. But there 
are still jobs for Cobol maintenance programmers. 
The Internet has grown tremendously and has fueled 
the need for Java and Html programmers. A final note 
- it may be worth examining differences between an 
on-line source and its hardcopy relative. 

A Very Informal Survey of Technical Books at a 
Mega Bookstore 

While the following study was lightly undertaken, the 
results support those found in the study of jobs from 
on-line sources. A mega bookstore in a New York 
City suburb was visited. Their extensive technology 
section carried books on a variety of programming 
languages. In this survey the number of shelves per 
books of a given language was counted. A shelf held 
about 35 books. Results are shown in Table 2, 



TABLE 2 



Cobol 


Visual 

Basic 


C++ 


Java 


Html 


0.5 


7 


8.5 


13 


4 



For those who believe that there will always be some 
books available for any particular language, the author 
offers the following observation: this store carried no 
Pascal books, neither on shelf nor in the computerized 
lists of books to stock. 

A Survey of Programming Languages Offered in IS 
Programs 

Through the listserv of ISWorld, a survey was emailed 
to 2415 recipients. 90 responses were received from 
79 institutions. Results are shown in Table 3. 



Visual Basic has the most offerings, followed by 
Cobol and Java. Html and C++ have the least. Note 
that the course offerings in this survey do not 
proportionately match job advertisements. It may be 
argued that academic programs respond slowly to a 
rapidly changing business environment, but the 
demands for periodic review and assessment of IS 
programs discussed previously in this paper suggest 
there are more compelling reasons for this 
discrepancy. Some of these reasons are considered in 
the next section. 

The Mission of a Particular IS Program and Its 
Student Body 

The author teaches at a small private college in a New 
York City suburb. While the college has a School of 
Business that houses the IS department, it is primarily 
liberal arts and sciences. With an open admissions 
policy, the college attracts many part-time students 
who tend to be older than the norm. These are non- 
traditional students; many hold jobs. Many are the 
first in their families to go to college. There are 125 IS 
majors from an undergraduate population of 4000. 

The IS faculty examine the language offerings 
regularly. In addition to the periodic review required 
by Middle States, every department at the college 
invites an outside evaluator to examine their programs 
and offer recommendations for improvement. This is 
done every four years. In 1995 we required a two- 
semester sequence in Cobol; this was changed to one 
course in Cobol and one in Visual Basic. 

The IS program covers 10 courses. The Cobol and 
Visual Basic courses are taken after a one-year 
sequence of computer literacy courses. IS is a difficult 
major for many students. Of all the courses, the two 
programming courses are most challenging. In 
particular, the Cobol course acts as a gatekeeper. 
More than 25% of the students attempting that course 
do not pass. Those that pass Cobol almost always pass 
the other IS courses. 

We have considered the more popular languages with 
higher career potential: C++, Java, Html. Thus far we 
have decided to continue with Cobol and Visual Basic. 
Our reasons are as follows. Cobol is not yet a dead 
language. There are jobs and books available. In 
addition, it is more common for an IS program to offer 
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Cobol than it is to offer C++ or Html, and as common Cobol is a very structured language and teaches a 
to offer Java. structured programming style. It is very concrete and 

students find it less difficult than the more abstract 



TABLE 3 



Course 


# Institutions 

Offering 


# Courses 

Offered 


Required 


Elective 


Cobol 


43 


50 


32 


18 


Visual Basic 


65 


69 


56 


16 


C++ 


40 


45 


36 


26 


Java 


48 


50 


31 


31 


Html 


46 


46 


25 


24 



Notes: 

a. In some cases a course is required or elective but 
offered by another department. 

b. In some cases a course is one of a group of which 
a few are required. 

c. In some cases a course is required in one track and 
elective in another. 

d. In some cases a colleague noted that a course was 
offered, but didn’t note whether it was required or 
elective. 

e. Html was often offered as part of a course. 

f. In cases where there was more than one response 
from an institution, responses sometimes conflicted. 

g. A prize goes to a New England institution that 
offered and required one course that covered every one 
of these languages. 

languages such as C++ or Java. A Computer Science 
program at our college offers C++ and Java. The IS 
program often attracts Computer Science students who 
find their languages too difficult and manage more 
easily with Cobol and Visual Basic. At this point in 
time, if we were to substitute C++ or Java for Cobol, 
the course might become more than a gatekeeper, it 
might become a wall scalable by very few students. 
We don’t want to decimate our majors. On the other 
hand, we want to offer our students courses they will 
find valuable for career preparation. Cobol remains a 
useful language. It helps to train students in the rigors 
of programming, introduces them to structured 
concepts, and provides a concrete vocabulary that is 
more easily understood than the more abstract 
programming languages. 

A final consideration is that our students do not 
typically become programmers. While we want them 
to understand what programming in a 3GL is about, 
we are not terribly concerned with a decreasing job 



market in Cobol. The few students who show high 
aptitude for programming are encouraged to take C++ 
and Java as electives. 
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EXPERIENCES ON USING A 
BUSINESS GAME AS PART OF AN 
INFORMATION SYSTEMS COURSE 



Timo Lainema 

Turku School of Economics and Business Administration 



ABSTRACT 

This paper describes an experimental teaching case used in an information systems course of a business university. 
In this teaching case the students were taught through the use of a real-time processed business game how typical 
business processes unfold. We were especially interested in giving the students perception on the holistic structure 
of organisationalprocesses and the balance between different functional demands. Thecaseexercise confrontedsome 
problems, both technical and organisation. This paper describes both the theory supporting the use of computer-based 
learning environment, the tools used in the case exercise, and the student survey results from the teaching case. The 
results of the study are preliminary and further research about the topic needs to be done in the future. 



INTRODUCTION 

Today, the information needs of decision-makers are 
likely to change rapidly. Instead of improving 
information about what goes on inside an organization 
decision-making will increasingly require information 
about competitors, new producing technologies, new 
delivery channels, and so on. The development of 
decision-making skills of employees in any business 
organization is a major challenge for businesses. In this 
process of continuous changes the teachingmethods play 
a considerable role. Besides of the changes in different 
business environments the higher-level business 
education is in a state of change. A ram and Noble (1999) 
argue that business schools are not adequately preparing 
students to understand and cope with the levels of 
ambiguity and uncertainty they will inevitably face when 
they take up positions in organisations. They believe that 
this is occurring because the models of teaching and 
learning that dominate academic practice are those that 
are appropriate to the stable, predictable aspects of 
organisational life and do not include the paradoxical 
and unpredictable. 



We feel that this same demand concerns also both to 
information systems professionals and information 
systems education. Information systems professionals 
are often in positions where it is essential to deal with 
issues concerning the interest oforganisationsas whole. 
Thus, these professionals need to understand the holistic 
structure of organisational processes and be able to 
balancebetween different functional demands. Lee etal. 
(1995) note that the IS industry will demand IS 
professionals with knowledge and skills in technology, 
business operations, management, and interpersonal 
skills to effectively lead organisational integration and 
process reengineeringactivities. Thus, the requirements 
for IS professionals are becoming more demanding in 
multiple dimensions, particularly in the areas of business 
functional knowledge. 

In this paperwe are discussing an experimental training 
case where students in an information systems course 
were to deal with business process identification through 
using a business game model. The experiment was a 
simple one andconfronted some setbacks. However, we 
feel that the experimentation as one way of arranging IS 
education is worth reporting. 
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THEORY SUPPORTING THE USE OF 
BUSINESS Games in teaching 

Computer simulation involves experimentation on a 
computer-based model of some system (Pidd, 1998). The 
model is used as a vehicle for experimentation, often in 
a ‘trial and error’ way to demonstrate the likely effects 
of various policies. Those policies, which produce the 
best results in the model, would be implemented in the 
real system. According to Bunge (1973) the purpose of 
using simulations is to gather understanding of the 
original object by studying the behaviour of the 
simulation: without analogy there might be no 
knowledge; the perception of analogies is a first step 
towards classification and generalisation. On the other 
hand, Bunge (1973) warns us about the inability to 
distinguish analogy from equivalence. Bunge mentions 
that an illusion of perfect formal analogy can be 
produced only in special cases. 

Business game is a special type of simulation in which 
human participants act as decision-makers within the 
framework of the system being simulated (Naylor, 
1979). Ju and Wagner (1997) mention that the nature of 
business games can include decision-making tasks, 
which pit the player against a hostile environment or 
hostile opponents. Business games have the nature of 
strategy games, but usually are very terse in their user 
interface. Other types of managerial simulations are 
resource allocation games in which the player or the 
players have to allocate resources to areas such as plant, 
production, marketing, and human resources , in order to 
produce and sell goods. 

Computer-based learning environments (CBLE) like 
business games - build on a long tradition of experiential 
education theory (Isaacs and Senge, 1992). This theory 
points to the significance of learning through direct 
experience opposed to learning through ‘instruction ’. In 
experiential education theory learning is said to occur 
through the resolution of conflictsover different ways of 
dealing with the world. 

Kolb (1984) describes three slightly different models of 
experiential learning. The model Kolb calls Lewinian 
model is an integrate d process th at begins with here-and- 
now experience followed by collection of data and 
observations about that experience. The data are then 
analysed and conclusions of this analysis fed back to the 
actors in the experience for their use in the modification 



of their behaviour and choice of new experiences. Thus, 
in this model learning is conceived as a four-stage cycle 
shownin Figure 1. Immediate concrete experience is the 
basis for observation and reflection. Observations are 
assimilated into a theory from which new implications 
for action can be deduced. Implications or hypotheses 
then serve as gu ides in acting to ere ate new experiences. 

FIGURE 1 

THE LEWINIAN EXPERIENTIAL 
LEARNING MODEL BY KOLB (1984) 
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Kolb (1984) notes that the emphasis is on here-and-now 
concrete experience to validate and test abstract 
concepts. Kolb argues that immediate personal 
experience is the focal point for learning. This 
experience gives life, texture, and subjective personal 
meaning to abstract concepts and at the same time 
provides a concrete, publicly shared reference point for 
testing the implications and validity of ideas created 
during the learning process. The second emphasized 
aspect'of this learning model is that training is based on 
feedback processes (Kolb, 1984). The information 
feedback provides the basis for a continuous process of 
goal-directed action and evaluation of the consequences 
of that action. 

Also Argyris and Schon (1978) have described this 
process with slightly different terms. In their model the 
process moves from discovery ofproblems, to invention 
of solutions, to production of solutions in action, to 
reflection on the impact of these actions, and then back 
to discovery. Isaacs and Senge (1992) argue that if this 
learning cycle operated effectively, new insights about 
the real world would be continually discovered and 
embedded in improved mental models. Decisions would 
be invented based on new mental models, those 
decisions wouldbe enacted and then outcomes would be 
reflected upon to produce new insights. 
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FIGURE 2 

CBLE LEARNING CYCLE BY 
ISAACS AND SENGE (1992) 
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Isaacs and Senge (1992) mention that the designers of 
CBLEs argue that the ideal learning cycle fails to ope rate 
effectively in organizations because of limits at each 
point in the cycle (these limits affecting the ideal 
learning cycle are embedded to the cycle in Figure 3.)- 

1. Decision-makers have diverse and typically tacit 
mental models, making the development of 
strategiesa process of negotiation among competing 
recommendations, not a rational comparison and 
testing of alternative assumptions. 

2. Delays between when decisions are made and their 
impact may be very long, often many years for 
strategic decisions. 

3. Additional delays between action and perceived 
consequences arise because of the time needed to 
collect, disseminate and interpret data. 

4. Actions taken in one area may have significant 
effects in distant parts of the system, but these 
effects may obscure to the original actors. Thus, 
decision-makers cannot see the consequences of 
their decisions. 

5. Differences in mental models held by decision- 
makers can lead to widely different interpretations of 
available data. Furthermore, the sources of the data 
may be rarely investigated and tested. 

6. Environmental factors, such as uncontrollable and 
unanticipated changes in markets, economic, 
conditions, or competitor actions inevitablealter the 
impact of decisions. This is true especially over the 
long time spans relevant to organizational actions. 



Isaacs and Senge call these factors Confounding 
factors. 

As the CBLE is placed to the previous ideas this results 
in the cycle presented in Figure 3 (Isaacs and Senge, 
1992). In this figure Isaacs and Senge use the term 
‘virtual world’. This figure illustrates us how a virtual 
world or a CBLE can turn visible the learning limits 
obscure in real world. Thus, CBLEs provide a rapid, 
unambiguous, and systemic feedback on actions taken. 
They provide a relatively low-risk setting in which 
differencesin mental models can be explored and tested. 
CBLEs can reflect back previously tacit assumptions and 
can provide insights into the nature of the complex 
interactions that determine the consequences of 
managerial decisions. 



FIGURE 3 

THE IMPACT ON LEARNING 
USING VIRTUAL WORLD 
(ISAACS AND SENGE, 1992) 
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Of course, Isaacs and Senge (1992) also note some 
difficulties and problems in the use of CBLEs limiting 
the learning experience. Researchers may typically 
operate in a way dev eloping vie ws, whic h they adv ocate 
unilaterally and see their task as imposing their view, 
and seek to protect their image as experts. By doing so 
they are undermining the effectiveness of the learning 
environment that they are seeking to foster. For example, 
the researcher may give little freedom to participants. 
This will create defensiveness and lessen the ownership 
of participants in the overall process. Isaacs and Senge 
divide the users of CBLEs to protective and reflective 
ones (Table 1). 
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TABLE 1 

PROTECTIVE AND REFLECTIVE USERS OF CBLES (ISAACS AND SENGE, 1992). 



Protective users of CBLEs 
CBLEs used to: 

Prove a point 

Keep assumptions hidden 

Validate own predispositions 

Make selective use of data 

Give predetermined answers, cover- 

up predetermination and remain 

unaware of doing so 

Act like an oracle 



Reflective users of CBLEs 
CBLEs used to: 

Increase practical experimentation 
Promote inquiry into ‘why’s* 

Challenge predispositions 

Encourage open testingand public criteria for testing 
Develop internal commitment to objectivity, actively 
encouraging inquiry into own views 

Experience the system and the assumptions in it as 
the cause of behavior 



REPRESENTING BUSINESS 
PROCESSES WITH A CONTINUOUSLY 
PROCESSED BUSINESS GAME 

There are two main types of simulation processing 
(Dictionary ofComputer Science, 1989): continuous and 
discrete event. The choice between these two in any 
particularcase is determined by the nature ofthe system 
to be simulated and the purposes that the simulation is 
intended to achieve. 

Discrete (event) simulation refers to an abstraction, 
which relies on a step-by-step representation (Wyatt, 
1975). In discrete event simulation it must be possible to 
view all significant changes to the state of the system as 
distinct events that occur at a specific point in time 
(Dictionary ofComputer S cience, 1989). The simulation 
then achieves the desired behaviour by modelling a 
sequence of such events, treating each individually. 
Continuous simulation refers to the abstraction of the 
real process being simulated to a set of mathematically 
continuous functions (Wyatt, 1975). A continuous 
simulation views changes as occurring gradually over a 
period of time and tracks the progress of these gradual 
changes (Dictionary of Computer Science, 1989). 

Whicker and Sigelman (1991) divide simulations into 
two categories, too. The first one is the batch-processed 
model, in which all behaviour subroutinesare spelled out 
beforehand, in the computer code. Thus, no human input 
is required during the simulation. Whicker and Sigelman 
call this method also as non-interactive model, meaning 
that there is no interaction during the simulation. The 
second one is the interactive model, in which the 
model’s performance periodically is adjusted to account 

224 



for input supplied by the modeller while the model is 
running. This requires the model to stop in midstream 
and pose a question to the modeller (Whicker and 
Sigelman, 1991). When the model receives the answer, 
it will proceed accordingly. Whicker and Sigelman 
describe how ‘business strategy games’ are processed: 
Typically, the player feeds information into a computer 
program and receives back a series of optional or 
additional data that are conditional upon the player's 
initial choices. The game proceeds through several 
series of these interactive, iterative steps. 

Huckfeldt et al. (1982) note that the choice of discrete 
time models rather than continuous time models has 
important substantive implicatio ns. However, they note 
also that the models employed should not be judged on 
the extent to which they replicate a social process, but 
rather on the extent to which they help us understand 
a social process. We feel strongly that in order to 
understand a process we have to be able to witness and 
see it, and more precisely to be able to interact 
with this process. The flow of decisions and their 
outcomes in games and competitions is seldom 
predictable. Their logic may find several different paths 
depending on the participants and be utterly 
unpredictable. Thisis why wethinkthat the main stream 
approach to business games the discrete-event or 
batch processing method lacks some necessary 
characteristics needed to describe the phenomena of 
business decision-making: the genuine interaction 

between the simu lation mo del and the game p articipants 
in unanticipated and unpredictable points of time. In 
discrete event models the human behaviour of the 
participants is inevitably an approximation of human 
decision-making. This is because the game participants 
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cannot within the simulated term - decide the exact 
point in time their decisions are executed. Instead, this is 
estimated and decided by the computer simulation. 

The treatment of time in business games remains little 
discussed. However, we fee 1 that the treatm ent of time in 
business games is an im portant asp ect, for exam pie, from 
the point of view of how the participants perceive the 
educational aspects of thetraining session. Thavikulwat 
(1996) argues that how time is treated in a computerised 
gaming simulation circumscribes the issues the 
simulation can address, the procedures that participants 
must follow, and the work that the administrator must 
do. The way time is treatedmay limit the issues that can 
be addressed and thatthe gaming simulation must allow 
a clear distinction between the short and long term. 
Thus, the pacing oftime is critical. For example, if time 
moves fast, participants’ decision-making depends more 
on gut-level attitude, less on time-consuming analysis. If 
time moves continuously, when participants decide can 
be more consequential than what they decide. 
Thavikulwat notes that in simulation, as in life, “time is 
of the essence . The management of time is an essential 
theme in the new business game construction described 
in this paper. 

Thavikulwat(1996) sees four different simulation drive- 
design possibilities (administrator-driven, participant- 
driven, clock-driven, and activity-driven simulation). A 
clock-driven simulation advances time in concert with 
the computer’s internalclock. Thavikulwat mentions that 
this design would be elegant in games that allow for 
interdependence among participants. Furthermore, 
Thavikulwat mentions one clock-driven business 
simulation, Chiesl’s ( 1990) marketing simulation . 

Chiesl (1990) quests for realistic business episodes into 
the university classrooms. He mentions that interactive 
computer terminals offer the possibility to construct 
dynamic simulations without a fixed time period or a 
specific number of required decisions. Chiesl calls this 
technique interactive gaming. Chiesl states that present 
(this is in 1990) business games are time fixed format 
games and their decisions are based on a predetermined 
decision interval, for instance, quarterly or monthly 
reports. According to Chiesl, this does not represent the 
working of today’s dynamic business world. Chiesl asks 
for continuous data input and output when students want 
it, not when game designers allow the students to input 
and output at some arbitrary discrete time format. An 
interactive business simulation would offer the students 
a more realistic environment than the fixed-time format 



business game. Thus, participants experience a business 
environment that has the appearance of being true and 
real. 

Chiesl (1990) argues that an interactive real time 
simulation offers a more realistic environment than 
standard business simulations because students are able 
at any time in the simulation to change their input 
variables without waiting until the next fixed time 
period. Furthermore, Chiesl argues that the students’ 
requests are instantaneously answered although this 
cannot always be the case as usually the outcomes of 
decisions take some time to materialize. Another 
exception to the majority of business games is that 
Chiesl’s interactive simulation does not require a 
specific set amount of decisions be input each time 
period. Thus, students can change one, some, all, or no 
variables whenever they choose. 

Besides of Chiesl (1990) the only two references we 
have found mentioning real-time processing are Patz 
(1990) and Lawrence (19 97). According to Patz (1990), 
simulations should become continuous rather than 
discrete processes. Decision rules should be entered at 
any time, and competitive results continue to occur 
whether or not any decision rule changes have been 
made. Patz notes that this will require a multitasking 
environment running several programs simultaneously. 
But why these kinds of structures have not been 
constructed before? Patz may give us one possible 
explanation: simulation purposes, for the most part, are 
decided by coding convenie nee rather th an pedagogical, 
conceptual, or theoretical relevance. 

We argue that to better describe the present business 
environment and processes, business games should 
include the influence and importance of time to be 
embedded in the businessgame. Besides of this, the new 
business game construction (REALGAM E) described in 
this paper includes a holistic view of business (allmajor 
business functions and stakeholders) to be represented 
forthe game participants. Furthermore, the construction 
includes the ability to configure the business game to 
describe different business environments. 

The time argument means that the connection between 
players, supply market, customers and capital market 
needs to be interactively processed. What is essential is 
the role oftime in simulating the time-bound business 
processes, decision-making, and the communication 
between the companies and different stakeholders. What 
is suggested here is a real-time or on-line processed 
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business game. Decision-making and results from the 
decisions made should be in an interactive on-line mode 
as they are in the real-world environment. In an on-line 
processed business game different business events and 
processes take place continuously and concurrently, and 
often in varying fre quency. T he participa nts steering the 
company perceive all the market events and internal 
processes on-line. What ever happens can be seen 
without a delay and action can take place instantly, if 
only the participants realize and needto do so. Thus, the 
game emulates the real world processes of business 
environm ents with the major exception that the internal 
simulation time is accelerated compared to the real 
world. In managing this kind of environment the 
participants’ ability to perceive processes and causal 
dependencies is essential. 

The on-line processed game (REALGAME) is 
developed at Turku Centre for Computer Science, and 
Turku School of Economics and Business 
Administration during 1997-1999. This new game 
construction has been used five times in teaching during 
1999 and 2000. REALGAM E is planned to describe the 
causal dependencies in decision-making better than the 
batch processed business games. Parts of the functions of 
REALGAME are introduced in Figure 4. 

In REALGAME there are 4-12 competing companies 
and the markets, suppliers and fundingorganisations are 



common to all participating companies. This game has 
been programmed with a Rapid Application 
Development (RAD) tool (Delphi) in Windows 
environment. Delphi enables the use of databases and 
supports user-defined inherited objects. Both of these 
resources have been noticed to be inevitable for the 
creationof REALGAM E. Databases are needed in order 
to record all the detailed business transactions taking 
place during the game run. Without the object oriented 
development environment it would not have been 
possible to create a truly configurable business game 
environment. 

In continuous game processing the functions are 
executed on a continuous and iterative manner. Different 
steps can enact simultaneously or apart. Different 
iterativedecision loopsmay occur in the same pace or in 
a different pace. The process is never ending and easing 
off the decision-making results most certainly in 
troubles. Furthermore, the participants are part of the 
gameprocesses. Thismeans that they see the changesin 
their environment to evolve on-line. We feel that this is 
a major difference if we compare continuous processing 
to batch-processing: the participants are organic part of 
the business processes and are able to witness and see 
them, and most of all - areable to continuously interact 
with this process. 



FIGURE 4 

PARTS OF THE FUNCTIONS OF THE REALGAME. 
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Actually, REALGAME does not operate in true real- 
time processing. True real-time processing would mean 
that each and every independent game object (e,g,, a 
customer or a supplier) would have its own internal 
Timer object, which would activate independently of 
everything else in the game and process all object 
specific tasks and processes. Very early in the 
development of the game this kind of true real-time 
processing was testedbut provedto be impossible to use. 
This is because oft he Windows operating system, which 
allows only a limited number of active Timer objects to 
be used at a time. However, the processing is continuous 
in the sense that: 

The game time is clock-driven, smallest increment 
in time being one hour. 

The participants are not tied to make decisions in 
specified points of time but can make decisions 
whenever they choose during the game. 

The decisions made in each point of time can be 
single decisions or several decisionsbut no decision 
batches are required. 

The participants may choose to run reports at any 
point of time. 

The participants see the internal and external 
business processes to evolve, e,g,, hour by hour, 
depending on the game parameters (explained 
below). 

According to de Geus (1988) one reason for putting 
mental models into computers is that in working with 
dynamic models, people discover that in complex 
systemscause and effect are separatedin timeand place. 
He mentions that to many people the insight to these 
causal phenomena is counterintuitive, thus, we are not 
able to find other trigger points than the most immediate 
causes to create the requested effects. The use of 
dynamic models helps us to discover other triggerpoints, 
separated in time and place from the desired effects. 
Another reason for the use of dynamic models, 
according to de Geus, is that by using computer models 
we learn what constitutes relevant information. For only 
when we start playing with these microworlds do we 
find out what kind of information we really need. If 
people are to learn about causal correlations and 
interdependencies, we need training tools, which can 
represent these causalities. 

These reasons put forward by de Geus have encouraged 
us to experiment with REALGAME in association with 
teaching the unfolding of business processes. We feel 
that REALG AM E can provide studentsperception about 



the causal both in time and structure dependencies 
between business activities, 

CASE CORPORATE INFORMATION SYSTEMS 

This paper describes oneparticularteachingexperiment 
using the real-time processed business game described 
above and a commercial process modelling software. 
The experiment was carried out in Turku School of 
Economics and Business Administration as part of an 
information systems course, 

Turku School of Economics and Business 
Administration (TuKKK, http://www,tukkk,fi) is the 
second largest business u ni versity in Finland, T uKKK is 
an independent university consisting of departments of 
Accounting & Finance, Management, Economics, 
Marketing and Languages. Information Systems Science 
(ISS) is part of the department of Managem ent. The role 
of ISS is twofold. Firstly, the main emphasis of the 
teaching given to the student studying IS as their major 
is to offer skills needed to specialise in professional 
information systems positions. Secondly, ISS gives 
several courses to a vast amount of students studying 
business economics as their major. The main emphasis 
of this teaching is to clarify the role of information 
systemsin organisational context and give basic skills in 
information technology utilization. 

One of the courses with emphasis on IS tools skills and 
aimed at the students studying other than ISS as their 
major is Corporate Information Systems (CIS), The 
aim of the course is to give the students readiness (a) to 
understand the importance of enterprise systems in the 
business operation context, (b) to be able to take 
advantage of enterprise systems, and (c) to resolve 
different business reporting needs. 

In spring term 2000 145 students enrolled on the course. 
The course included 14hours of lectures during a seven- 
week period with emphasis on the use of different 
computer tools commonly used in different business 
organisations. The themes covered on the course were: 

principles of creating queries and reports with a 

relational database application (6 hours), 

the use of a typical enterprise information system in 

the supply and delivery process (4 hours), 

the use of a project modelling application (2 hours), 

and 

describing business processes with a process 
modelling application (-2 hours), 
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The tool used in describing business processes was 
Process Guide (PG). PG is a commercial computer 
application developed by QPR Software Ltd 
(http://www.qprsoftware.com). PG is a tool that can be 
used to: 

Create business process models to illustrate the 
current status of operations. 

Monitor and measure performance in areas such as 
lead-time, costs and quality. 

Identify bottlenecks and simulate processes to 
analyse alternatives, predict operational changes, 
validate new plans and plan resource allocation. 

From the using stand of pointPG is a relatively simple 
program. Mainly, the users need to: 

know from three to sixgraphical flow chart symbols, 
be able to create a process flow chart using these 
symbols, 

name cases to be studied within the process chart, 
attach different measures(for example, costs or lead 
times) to case specific tasks defined in the flow 
chart. 

One of the aims of CIS is to give the students a holistic 
view of business processes and the role of corporate 
information systems in this holistic structure. Based on 
this aim the teacher of CIS (the author of this paper) 
decided to combine an imaginary business game case 
with theuse of PG. The teacher as a researcher- hoped 
to gain knowledge on how a real-time processed 
business game would suit to be used as a business 
process case description tool. This business game case 
would then be modelled with PG. Both the use of the 
business game and creating the process model with PG 
were done independently by the students. The students 
were told to carry out the assignment either alone or in 
groups of two students. 

The case was described, the tools represented and the 
assignment carried out as follows: 

Both tools used (REALGAME and PG) were 
introduced and used in the two-hour lecture. The 
game configuration used was as a netwo rk game, all 
the students in the class competing against each 
other. 

The assignment was introduced in the lectures and 
also included as a web page. The assignment 
included a detailed description of computer task 
steps to be taken in order to start the business game. 



Furthermore, the course pages included an eight 
page long guide on how to use PG. 

As the students independently carried out the 
assignment they first initialised the business game to 
be used in the computer they had logged in by 
starting a DOS batch file. 

Then the students were supposed to start the game 
market application and as the last step start the 
business game decision-making application. 

After they had played the game long enough to get 
an understanding of how the delivery process was 
processed in the game, they were supposed to 
illustrate and document a customer specific business 
game delivery process as aPG case. 

The students were supposed to return the PG case 
descriptions(the PG process flow file)to the teacher 
to be evaluated and given performan ce marks. With 
PG the students first created a process flow chart. 
For the tasks in the process flow chart the students 
entered case specific values to a certain process 
measure, the lead-time. The values for the lead-time 
were supposed to be taken from the processes 
simulated with the business game. After the values 
had been entered to the tasks the students were asked 
to browse through the differentanalysis tools of PG, 
e.g., a Gantt ch art. 

After all the previous steps the students were asked 
to return a questionnaire on how they experienced 
the use of the business game as a case description 
tool in a process modelling exercise. This 
questionnaire was filled out by individual students 
(even if the student was part of a group) and 
returned anonymously. 

The teacher tested the use of REALGAME beforehand 
in the computer classes. However, the test was too short 
in time to reveal that the capacities of the computers in 
use were too low. Overall, 21 students directly stated 
that they had technical problems during the game. 
Actually, we have every reason to believe that most of 
the students faced technical problems. This was due to 
the two computer classes used in this exercise. The 
computers in the classesproved to be much too slow for 
running the business game. 

RESULTS FROM THE 
TEACHING EXPERIMENT 

The analysis ofthe questionnaireproduced the following 
results. Tables 2 and 3 describe the results of the two 
closed questions. 71 students of the total of 1 31 students 
takingpart on the exercise returnedthe questionnaire. 56 
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of the students had previously taken part to some other 
business game training with some other business game. 
Most of these had played the old game of the school, 
which is a typical batch-processed total enterprise 
business game. The students all had previous experience 
from case exercises carried out in ISS and other courses. 
We thus assumed that the students were quite familiar 
with the strengths and weaknesses of case exercises. 

The results in Table 2 show that the students did not 
clearly prefer either the business game case or a text 
case. We find this result encouraging considering the 
technical difficulties the students faced during the 
experimentation. We also believe that the design of the 
experimentation was inadequate (as mentioned later in 
conclusions). As this was a preliminary test with the 
game in this kind of exercise we feel that by carefully 
developing the exercise it is possible to achieve better 
results. 

The results of Table 3 affirm the interpretation wemade 
from the results of Table 2. Table 3 shows clearly that 
students found potential in this kind ofexercise and it is 
probably that the technical difficulties affected the 
answ ers of T able 2 . 

Below we have collected some typical answers to the 
two open questions of the questionnaire. Some answers 
to the question . Your general comments on the use of 
the real-time processed business game? 

• With games you get a proper feel of the complexity 
of business decision-making! The game must be 
played, e.g., two days at a time in order the ‘*idea " 
to become clear. This was missed in this game and 
this is why understanding and evaluating this game 
is difficult. (International marketing, 3^*^ year) 

• It was needless waste of time to mix the game with 
this exercise. (Accounting, 3'^^ year) 

• Store management and production process well 
implemented but strategic decisions are not well 
enough presented. (Accounting, 2 year) 

• ... The phases of delivery process were better 
represented in the game than would have been in a 
text case. (Accounting, 3'^^ year) 

• Good from the stand of point of understanding the 
holistic process but the realism with the game was 
so and so. (Marketing, 2"^^ year) 



We should be able to play the game m ore in order to 
fully understand the process that the game was 
intended to describe. If the game functions better 
and the game would be played at thesame tim e with 
other companies it would be a good teaching tool. 
(Accounting, 4^^ year) 

• Quite good thing that one had to think more on eself 
when one made the process model description. I 
thinka text case is much simpler. (International 
marketing, 2"^^ year) 

• If the game only would work all right it would be 
just nice alternative “teaching method”. It would be 
a good thing if there were more response on the 
success of gaming as the game proceeds. 
(Information systems, 2"^^ year) 

• Helps visualise chains of tasks and through this the 
functioningas awhole. (International marketing, 2 
year) 

• The game crashes a lot but it clearly includes 
potential because it represents well the dijferent 
business functions. It also well describes the 
dependencies of functions. (Accounting, 2”^^ year) 

• The game did not adequately represent the 
successive dependencies of business tasks. All the 
functions were separate from each other behind 
menus. You have to in advance yourself piece 
together the order of the functions. I think that the 
game does not teach you anything: quite the 
opposite: it mixes your conceptions. (Accounting, 
year) 

• There certainly is potential as soon as the technique 
works all right and the system crashes are 
prevented. It is not meaningful to play the game 
alone with the computer. Only in group-use 
competing against others it is fun and interesting. 
(Marketing, 2"^^ year) 

• ■ There certainly is potential but the game was not 

best possible for this exercise. The process of the 
game was too lean as a case to be used in 
connection to PG. We would have needed a more 
detailed process to be represented with PG... 
(Accounting and Finance, 2"^^ year) 

• I think the game has its role in outlining the business 

reality although the real world is much more 
complex than this kind of (Accounting and 

Finance, 2"^^ year) 
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TABLE 2 

THE RESULTS OF THE QUESTION CONCERNING THE 
STUDENT PREFERENCE OF GAME OR TEXT CASES 



Question 


Text 


Game 


No 


Both 




case 


case 


stand 




An alternative for the use of a business 


N= 35 


N=34 


N= 1 


N= 1 


game case description could have been a 


49.3 % 


47.9 % 


1.4 % 


1.4 % 



text case description. Which one would you 
have preferred in this assignment? 



TABLE 3 

THE RESULTS OFTHE QUESTION CONCERNING THE 
POTENTIAL OF BUSINESS PROCESS TEACHING 

Question Yes No 

Did the real-time processed business game include teaching N= 49 N= 19 

potential to be used as a tool to clarify the flow of business 72.1 % 27.9 % 

processes? 



The answers clearly reveal two things. First, the time 
spend in the exercise was quite too short The students 
did not achieve required understanding ofthe complexity 
of the case company. Secondly, several students stated 
that the game should have been played in a competitive 
manner, companies fighting against each other. This 
certainly would have made the gaming much more 
interesting for the students and created enthusiasm 
among them. Several of the students already in their 
answers to this question mention that the game 
introduced business processes. 

Following some of the answer to the question: If you 

did think that the real-time game included potential 
to clarify businessprocesses describewhich processes 
can best be clarified? 

• Best can be clarified the offer — order — delivery - 
invoicing chain. ... Also the competition situation is 
illustrative if the game can be played ‘'a day at a 
time” against other companies. (International 
Marketing, year) 

Sales and purchases (Marketing, 2”^^ year) 

Store management and the importance of time in 
decision-making. (Business Law, year) 

The store management necessitated by the 
production process, and the importance of well- 
timed deliveries. (Accounting and Fin ance, 2 year) 



The delivery process, I guess, although I was 
confused by the perpetual changes in time and the 
changing of production shifts... Besides the 
situations did not follow each other so that the 
gaming would have been easy... All gaming without 
any longer absorption and, for example, theproduct 
to be sold is the ultimate culmination of theorisation. 
It is really hard to enter into the game effectively if 
you do not know anything about delivery and order 
management times, if you have not made 
acquaintance yourself to the processes of the 
company... (International Marketing, 2 year) 

The different phases in order — delivery process 
were easily to be discovered. Furthermore, the 
influence of purchases and funding to company 
operations were clearly expressed with the use ofthe 
game. I did not familiarise myselfwith the marketing 
process but the impression of it remained quite 
incomplete. Thus, there is potential mainly in order- 
delivery process, purchases, and the monetary 
process. (Accounting and Finance, 3^^ year) 

You immediately noticed that with a more 
competitive offer you get more orders. The 
complaints of the customers about late deliveries 
were realistic. (Accounting and Finance, 2"^^ year) 
The basic processes became very clear. E.g., 
receiving orders and delivering, and store 
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management and deliveries in connection to store, 
(International Marketing, 2""* year) 

... I think that the fast speed of the game and 
delivering the orders in time were challenging. On 
the other hand it was good that the game company 
functions were not taught before the playing but we 
were forced to ourselves realise what different 
functions to take care of when playing, (Information 
Systems, 2"^ year) 

Purchases, production, delivering, invoicing and 
funding are well represented. The real-time 
processing of the game increases the fascination and 
gives the game an intensive nature, I think the 
processes have been logically represented and they 
are diverse. This gives a good basis for th e game to 
be used as a training tool. Also taking under 
consideration the different tim e spans of receivable 
and payable gives depth and realism, (Accounting 
and Finance, 2"^ year) 

Many of the previous answers reveal that the students 
had realized the nature of the game company’s material 
process from raw material purchases to deliveries. 
Several of the answers also mention the influence of the 
time aspect on decision-making. Most of the answers 
clearly advocate the use of the game in clarifying 
business processes. This is probably the most important 
result from the experimentation encouraging us to 
further develop the gaming case for future ISS courses. 

CONCLUSIONS 

Although there is a lot of criticism in these answers we 
certainly see some encouraging issues. On the basis of 
the student opinions we are inclining to argue that areal- 
time processed business game has some advantages 
compared to text cases. A game can represent the 
complexity of a business much more realistically than a 
text case. Furthermore, a business game can reveal the 
dependencies between functions in a much more 
illustrative way than a text case. The experiment 
described in this paper left a lot depending upon the 
students to themselves get motivated and interested in 
thebusiness game. Theteaching experiment described in 
this paper faced some severe misfortune but still some 
positive results of the experiment can be perceived. The 
answers to the questionnaire were clearly twofold. Other 
students were definitely irritated by the technical 
problemsthey met during the experiment. The attitude of 
these irritated stude nts towards the gam e most p robably 
was distorted and we were not able to get objective 
opinions on the gaming from them. 



As we were expecting that a reflective stand (Table25 
September 2000 1 , Isaacs and Senge, 1 992) to the game 
exercise would increase the learning value of the 
experiment we were a bit disappointed. Thus, the 
students did not take thefull advantage of the possibility 
to open testing andpractical experimentation . However, 
we are inclining to believe that this was because of the 
limited time for the exercise and some obvious 
shortcomings in the exercise design. It must be ensured 
that the time used to the exercise is long enough for the 
students to really get familiar with the processes. This 
probably means that the exercise must be carried out like 
in conventional business game training so that the 
students are all gathered together at the same time and 
place to compete against each other. Securing the time 
needed for the exercise will also bring along the 
competitive situation between the participants, which 
will further enhance the student interest in the game. 

Despite several studentsfacedtechnicalproblems as they 
used the business game they did not preferclearly either 
a business game case or a text case. This will encourage 
us to further experiment with REALGAME in similar 
exercises. In fact, we believe that a more thoroughly 
designed game case exercise will produce both better 
learning results and p articipant m otivation, W e strongly 
feel that the results achieved here speak for continuing 
with this kind of research. More solid evidence of the 
applicability of gaming will be represented in the future. 

As an overall conclusion from the experiment we argue 
that there is educational potential in the use of 
simulationsdescribingbusiness process flows. However, 
the teaching session has to be carefully briefed and 
motivated. The exercise mustbe challenging enough and 
not to underestimate the skills of the participants. For 
more information about REALGAME business 
processes see http://www.tukkk.fi/~tlainema. 
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ABSTRACT 

The purpose of this article is to draw a distinction between the use of group work and subgroup work in the delivery 
of online courses, and along the way to pose the question ‘‘What should be the objective of the instructor? ” To do 
this, four different models of online course delivery used in the teaching of IT and IS courses are described. Each 
model is distinguished from the others by the varying levels of group work (low to high) and subgroup work (low to 
high) used as an essential component of the learning process, and the advantages and disadvantages of each model 
are discussed. The paper concludes by suggesting that the objective of the instructor may play a hidden but 
substantive role in the selection of the model to be used. 



INTRODUCTION 

An appropriate definition of collaborative learning that 
will suffice forourpurposes is .a learningprocess that 
emphasizes group or cooperative efforts among faculty 
and students. It stresses active participation and inter- 
action on the part of both students and instructors. 
[Bruffee 1984], quoted by [Hiltz 1994]. 

Collaborative learning is hardly a new topic. The 
importance and relevance of social interaction to an 
effective learning process has been stressed by many 
theorists from [Vygotsky 1978], through advocates of 
situated learning such as [Lave and Wenger 1991], and 



has been confirmed by many more recent researchers 
and practitioners. 

An up-to-date review of the research and the long history 
of peer/collaborative learning can be found in [McKeachie 
1 999]. A small but select annotated bibliography on col- 
laborative learning can be found at [Planck et al 1999]. 
An excellent list of strategies and tips for those interested 
in introducing collaborative learning can be found in 
[Davis 1993]. 

When students and instructors are linked by computers, 
the groups become electronic collaborative learning 
groups (ECLGs). Several studies appear to show that, 
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particularly for more mature students, such a learning 
environment can be more effective than traditional face- 
to-faceclassroom sessions. See for example [Hiltz 1 994, 
Haras ini 1995] . 

Other benefits of group work, or collaborative learning, 
have been widely described in the literature. For 
example, group work is seento enhance critical thinking 
[Gokhale 1995], and can assist in the retention of 
minority students [Berry 1991]. 

The phrase “group work is usually used within an 
academ ic context in cases where the students are broken 
up into small groups; however, it is also used in cases 
where all of the students in the class work as a group. It 
is useful to distinguish between the two. This paper uses 
the term “group work to mean an activity where all of 
the students in a class can interact (and thereby assist 
each other), and “subgroup work to mean an activity 
where interaction occurs only within small distinct 
groups (and thus subgroups could be perceived to be 
competing against each other). This, the differentiation 
between “group and “sub-group is not at all a matter 
of the size of the group, but of whether the students 
perceive there to be competition between groups. 

The phrase “online delivery is used to refer to a course 
which is primarily web-based, with course materials and 
other items being made available over the Web to 
students connected from remote locations. 

THE INSTRUCTOR’S OBJECTIVE 



When the pedagogy to be used for theonline delivery of 
a particularcourse is discussed, many topics are debated, 
but the objective of the instructorisrarely amongstthem. 
For it is assumed, firstly, that the instructor is there to 
assist the students to learn, and secondly, that since this 
is so obviously the case, no further consideration of the 
matter is warranted. 

In the academic context, all of the students share a 
common objective to master the material sufficiently 
to ensure a good grade. In the “real world outside of 
academia, it is generally the norm that people with a 
common objective will work together as a team, either 



formally or informally. Only where there is an element 
of competition where the success of one might mean 
the failure of another is this notusually the case. 

Now, it may be argued that the students do not in fact 
share a common objective rather, each student has the 
unique objective that he or she should get a good grade, 
rather than the whole class. 

What then should be the instructor’s objective? 
Suppose that the instructor has the choice of using two 
different methodologies, both of which tend to produce 
results in the form of bell curves with the peaks 
occurring in the Pass or Credit ranges; but that in one, 
the peak is very pronounced, with very few students 
achieving results in either the High Distinction or Fail 
ranges, whereas in the other the peakis less pronounced, 
with the final range of resultsbeing more widely spread. 
Suppose further that the difference is not a result of the 
marking scheme alone, but a true reflection of the 
students' learning. In such a case, which of the two 
methodologies should be preferred, and why? The 
current authors are not aware of any literature directly 
addressing this question ; a paper discussing this issue in 
some depth is in preparation [Roberts, 2001]. 

It is at least arguable that faced with such a dilemma the 
instructor should strive for the good of the class as a 
whole, rather thanfor the good of particular individuals. 
This has obvious parallels in the realm of political 
philosophy, where to take an admittedly simplistic 
generalisation the theoretical Socialist will work for 
the good of the population as a whole, in the belief that 
this is most likely to benefit the majority of individuals, 
whereas the theoretical Capitalistwill work for the good 
of each individual, in the beliefthat this will benefit the 
population as a whole. In what follows, therefore, we 
will refer to the former as a type-S choice, and the latter 
as a type-C choice. 

This paper looks atfourmodels ofcollaborative learning 
that fall on either side of this political divide. 

THE FOUR MODELS 

The four appro aches, all in current use in the delivery of 
online Information Systems courses at Central 
Queensland University, are characterized in Figure 1. 
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FIGURE 1 




Level of Sub>Group Work 



The traditional model is characterized by little or no use 
of subgroup or group work in the delivery of the content 
or the assessment of students. The radical model uses 
both subgroup and group work to almost the maximum 
possible extent. Sitting between these two, are the naive 
model, which uses subgroup work but no group work, 
and the standard online model, which uses group work 
but no subgroup work. 

The Traditional Model 

The traditional model is used in probably 95% of classes 
where face-to-face teaching in the form of lectures and 
tutorials is the norm. This model therefore tends to be 
replicated almost by default when those same 
courses are delivered online. Group work is minimal, 
and students are expected to learn in a “sage on the 
stage -type scenario. Indeed, the translation to online 
delivery often results in less collaborative learning, since 
the ad-hoc informal groups that may have been used in 
tutorial sessions are now dispensed with; consequently, 
little or no learning takes place in a group setting. 
Individual students receive feedback and criticism 
primarily from the instructor(s). In some cases this is 
wholly in the form of returned items of assignment; in 
others, a varying amount of feedback may be given via 
email or other means. 

There is therefore little or no opportunity for students to 
learn from each other, and the learning process is heavily 
reliant on the knowledge and teaching skills of the 
instructor. 



Interestingly, even where the traditional method is used, 
students have a tendency to form their own informal 
study subgroups. This appears to be a more common 
practice in some cultures than others, being particularly 
predominant amongst students from countries in the 
Southeast Asian region, such as Singapore and Hong 
Kong. Often it can happen that members of such 
“unsanctioned subgroups end up being penalized 
because of the similarity of their assignment 
submissions. Thus, the traditional model can in fact be 
antagonistic to the idea of collaborative learning. 

The traditional model can therefore be categorised as a 
type-C choice; whereas this may be as the direct result of 
a conscious decision by the instructor, it is at least 
equally as likely to be the result of a default to the 
currently-predom inant paradigm. 

The Naive Model 

The naive model involves the use of sub-groups, butnot 
the group as a whole. The most common form of the 
naive approach^is for students to be placed into small 
subgroups perhaps three or four students per 
subgroup for the purpose of working on and 
submitting one or more items for assessment. 

Thus, the instructor can be claiming to be using group 
work, but again in an ad-hoc fashion . Often, the method 
is used prim arily for reaso ns of expe diency in a class 

of 100 students, for example, having to mark 25 
assignment submissions is often seen to be a far 
preferable and less daunting task than marking 1 00. On 
the positive side, using subgroup work such as this may 
allow the assessment task(s) to be of a more realistic 
complexity than could otherwise be the case. 

The naive model allows the students to claim at 
subsequent employment interviews that they do indeed 
have experience of workingin groups. And it is true that 
some students may have gained educationally from the 
experience of working in such groups. However, the 
lack of defined guidelines as to how the groups are to 
operate, and their use in the context of a single 
assignment submission, means that in many cases the 
work will have been carried out primarily by the more 
able student(s) in the group, and little real benefit will 
have been obtained. 

The naive model can be enhanced so that group work 
becomes an integral and more important component of 
the course. To do this, consideration needs to be given 
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to the composition of the groups, the rules under which 
they operate, the methods to be used for assessing 
individual contributions, and other similar factors; see, 
for example, [Davis 1 993, Emerson 1997, Gregory 1 994, 
Wolf 1994]. 

In most cases the nai ve model can also be categorised as 
a type-C choice; however, if more enhancements are 
added to increase the value of the subgroup work, the 
categorisation becomes less clear. 

The Standard Online Model 

The standard online model uses group work (but not 
subgroup work) almost as an accidental by-product of 
the technology. The inclusion of online discussion 
groups allows individual students to pose questions and 
receive feedback from all other members of the group. 
In this way, students are enabled to learn as much from 
each other as they do from the instructor. This is one of 
the most commonly quoted advantages of online 
distance education: see for example [Jones 1996a, 
1996b]. 

Apart from online discussion lists, other forms of 
interaction from the group as a whole are possible. For 
example, provision may be made for anonymous 
feedback to be provided throughout the semester, thus 
allowing individual students to express comments 
without fear of retribution, andallowing the instructor(s) 
to glean valuable information by which to improve 
future offerings. Such a feedback mechanism has been 
described by [Svensson 1999]. 

The standard online model can be combined with the 
naive model, so that subgroups are used for particular 
items, but questions can be posed by, and feedback can 
be gained from, all m embers of the group. In such cases 
the resulting model would be located close to the centre 
of the diagram in Figure 1. 

Despite the opportunities for whole class interaction, in 
practice the standard online model remains a type-C 
choice, with little or no formal collaborative subgroups 
being used, and little or no group assessment work. 

The Radical Model 

Whereas both the naive model and the standard online 
model use subgroup and group work to a certain extent, 
the radical model uses both subgroup work and group 



work to almost their fullest extent, both as a means of 
learning and as a means of assessment. 

In the radical model lectures are dispensed with entirely. 
Instead, students are formed into subgroups, andlearn by 
interacting amongst themselves, and using the vast 
amount of existing Web-based resources, with the 
academic staff member(s) providing guidance as and 
when required. A detailed description of the radical 
model can be found in [Rom m & T aylor 2000]. 

Distinguishing features of the radical model include: 

a video sent out to all students prior to the 
commencement of semester explaining “the way the 
course works ; 

minimal traditional instruction from the academic 
staff; instead, students are expected to use the set 
text, and make extensive use of search engines and 
other facilities to seek outresources available on the 
Web; 

compulsory use of the course mailing list for 
communication; 

online electronic presentations prepared by the 
students themselves, each based on the topic forthat 
week; 

the allocation of students into subgroups, each of 
which is responsible not only for providing an 
electronic presentation at some point during the 
semester, but also for responding critically to all 
other such presentations. 

The online student subgroup presentations are expected 
to highlight the main points of that week’s topic, to 
explain and, where necessary, defend these points, and 
to suggest issues for further thoughtthat will engage the 
rest of the students in a critical analysis and discussion 
of the presentation. A one page summary, which can be 
in the form of a formal abstract, precedes the 
presentation, followed by a well-argued analysis/critique 
extending overtwo further pages (screens). Students are 
welcometo use additional sources and links as required. 

Students are assessed not just for their subgroup 
presentation but also for their comments about other 
presentations. Each subgroup presentation is also 
assessedon the quality of the discussion that follows;for 
this reason, it is important to the students that their 
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electronic presentations are well crafted, thought 
provoking, and intriguing. Typically, by the end of 
semester, students will have received over one hundred 
inputs on their work from other students in their 
subgroup, other subgroups, and the lecturer. 

Different assessment criteria may be used for 
example, for the electronic presentation, clarity and 
structure of presentation, originality of ideas, and ab ility 
to substantiate arguments by relevant data; for other 
contributions, understanding the arguments that are 
made by other presenters, linking them to the relevant 
literature, and making pertinent critical comments about 
these arguments. 

In the last week of term, students are invited to submit a 
recommendation in writing on the performance of other 
students within their subgroup. The lecturer considers 
the subgroup’s recommendations when allocating 
individual marks to members of that subgroup. A 
student that a subgroup decides did not contribute 
sufficiently may as a result suffer a reduction in mark. 

The students’ final marks are basedon a combination of 
their subgroup work throughout the semester, and their 
performance in a closed-book end-of-semester examina- 
tion. 

Amongst many real advantages of the radical model may 
be listedthe emphasis on subgroup-work, the need to use 
real-world skills both for effective communication and 
research, and the significantly lower demands on staff 
time than with most other models. However, students 
need to adapt early to the demands of the model (the first 
presentations are made as early as week three or four of 
the semester), and the model is perhaps more appropriate 
for postgraduate and later-year undergraduate students 
rather than recent school-leavers. 

The radical model is the only one of the four models 
described here which falls clearly within a type-S 
categorisation. However, although this is undoubtedly 
true within the learning context, the assessment can be 
skewed toward either type-C or type-S categorisation 
depending upon the weight afforded to the final 
(individual) examination. 

CONCLUSIONS 

Four different models of online course delivery have 
been described which vary according to the extent to 
which two forms of collaborative learning, group work 



and subgroup work, are employed as both learning and 
assessment tools. 

The radical model uses both group work and subgroup 
work to a significant extent. However, even in cases 
where the radical model is employed, it is usual to have 
an individual end-of-semester examination as a 
significant component of assessment. 

It is instructive to conclude by returning to the question 
raised at the beginning, the objective of the instmctor. 
In the traditional model, each student is expected to learn 
largely on their own; this is likely to be hardest for the 
least able students, and easiest for the most able. The 
traditional method therefore can be expected to produce 
a significantly wide range of results. Therefore, if the 
instructor is of the view that a wide range of results is 
preferable to a narrow range, the traditional model is 
likely to be fevoured. 

As a greater amount of collaborative learning is 
included, as with the nai’ve and standard online models, 
so are the least able students able to benefit more readily 
from the increased interaction w ith theirpeers. Thus, the 
range of results is narrowed slightly, while it seems 
probable that the mean has remained the same, or 
perhaps increased slightly. The naive model is likely to 
be chosen by the instructor not because of this, but more 
likely because of the belief that it is benefic ial to expose 
the students to some form of collaborative learning. 

The standard online model is most likely to be selected 
primarily because of the capabilities of the existing 
technology, rather than for any theoretical pedagogical 
reasons. 

When the extent of group work and subgroup work is 
increased to a very high level, as with the radical m odel, 
it is to be expected that the range of results will tend to 
diminish, while the overall pass rate will tend to 
increase. This model (or something like it) is likely to be 
chosen by instructors seeking the maximum amount of 
interaction amongst their students, and is the only one of 
the four models to fall clearly within a type-S 
categorisation. 

Thus, it can be seen that in practice the choice of the 
model to be used for a particular course is not only 
dependent upon strictly pedagogical issues. It is likely 
also to be determ ined to a significantextent by the prior 
attitude of the instructor toward collaborative learning, 
and in particular whether he or she believes that a wide 
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range or a narrow range of results is preferable . Thus, in 
the mind of the instructor, the type-C or type-S 
distinction plays an important (but often unconscious) 
role, 
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ABSTRACT 

This paper considers how active learning events in the form of problem-solving/discussion events can becombined 
with the traditional lecture to better support good teaching and s tudent learning in IS education, A single case study 
is used to illustrate and evaluate the effectiveness of small group problem-solving and discussion in large classes, 
for student learning. The results are presented and implications for effective teaching are discussed. 



INTRODUCTION 

Much of university education is based on the theory that 
students will learn if information is transmitted during 
lectures or if they do things in practicals or seminars 
(Ramsden, 1992). Although these are not necessarily 
inaccurate conceptions of teaching, they represent 
narrow visions of teaching. By contrast, good teaching 
is viewed as a process of working cooperatively with 
students to change their current understanding (ie. 
change the way in which students see and use the 
knowledge they have); it allows students to actively 
engage with subject content in a way that is likely to 
enhance understanding. Ramsden further suggests that 
enhanced understanding is most likely to occur if 
teaching methods encourage student activity, problem 
solving and cooperative learning. Students are then 
more likely to engage with the material at higher 
cognitive levels including detailed analysis and 
understanding. 

For teaching to be successful, it is important that 
teachingconceptions and practice be congruous with the 
goals of education. In higher education goals are 
articulated as concept use, deep knowledge and 
understanding, and application (Ramsden, 1992); in IS 
education these goals include concept/use (compre- 



hension and ability to use knowledge when asked) and 
detailed understanding and application (selection ofthe 
right thing and using it without hints) (Davis et al., 
1997). The IS '97 Curriculum report further identifies 
teaching methods such as laboratory work and project 
participation and presentations involving explanation, 
demonstration and criticism as congruous with the goals 
of IS education. Nevertheless, IS education like higher 
education, is criticised for failing to produce graduates 
who have achieved the goals and objectives of IS 
education. The concern then arises whether IS education 
is providing the right type of education for the future IS 
professional. A goal of IS education is to produce 
graduates who are equipped to function in an entry-level 
position in the organisation and who have the basis for 
continued career growth (Davis et al., 1 997). Hence, Lee 
et al (1995) argue that the IS curriculum must be driven 
by a clear vision of the career path for graduates. Th is 
implies that IS graduates must not only have the skills, 
knowledge and understanding appropriate to their 
specialisation, but must also be life-long learners, ableto 
question, to think critically and independently and to 
learn. If students are to develop these abilities (e.g., to 
think critically, and reason logically, creatively and 
flexibly in a variety of new situations) then educators 
need to examine both how and what they teach 
(Chalmers & Fuller, 1996). 
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Studies in IS education have tended to focus on the 
development of curriculum that meets the needs of 
industry (e.g., Davis et al., 1997; Gupta & Wachter, 
1998) and defining an appropriate balance between 
technical expertise and business knowledge (e.g., Lee et 
al., 1995). While educators have recommended useful 
strategies for IS teaching (e.g., Davis et al., 1997), few 
have evaluated the effectivenessof these strategies in the 
context of IS teaching and student learning. A Ithough 
there are many factors that impact teaching and learning 
(e.g., context, personal factors, prior knowledge), this 
study emphasises teaching that encourages active 
engagement in the large class lecture setting. This study 
is expected to enhance current understanding of how the 
lecture setting can be augmented to better support the 
goals of higher education (and in particular, IS teaching), 
by evaluating the effective ness of sm all group problem- 
solving/discussion events in the large class setting. Such 
study is particularly relevant since the lecture setting, 
despite criticisms, remains a dominant form of face-to- 
face teaching in higher education. 

ACTIVE LEARNING AND THE CONTEXT 

OF TEACHING IN HIGHER EDUCATION 

Higher education utilises the lectu re as the m ain context 
for university teaching in large classes. While research 
has determined lecturesto be most effective for coverage 
and transmitting information, (e.g., Bligh, 1998), the 
lecture is strongly criticised for its inability to readily 
support the goals of higher education, that is, 
understanding and the application and evaluation of 
ideas (Jenkins, 1994). However, where active learning 
techniques are used, the lecture can be made more 
effective in terms of retention, transference to new 
situations, problem solving, and critical thinking 
(Horgan, 1999). Indeed, Biggs (1991) argues that the 
more ways in which a student engages in task-related 
activities, the stronger the learning. While the lecture 
context traditionally implies teaching as transmission, 
teaching for learning can be augmented by using 
mechanismsthat actively engage students in the learning 
process. 

Active learning is based on the assumption thatlearning 
is by nature an active undertaking, and that different 
people learn in different ways (Meyers & Jones, 1993); 
it presumes that students learn best by doing. Active 
learning provides opportunities for students to talk and 
listen, read, write and reflect on course content through 
problem-solving exercises, small group discussions, 
simulations, case studies and other activities. Biggs 
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(1999) also suggests that active engagement in the 
learning process encourages the less academic student to 
employ high-level engagement techniques such as 
theorization, reflection, applyication, which are more 
naturally adopted by the more academic student even if 
the teaching method is more passive. 

The followingcase study illustrates how the passive, less 
rigid, transmission-oriented environment of the lecture 
can be modified to encourage more active dialogue 
between student and teacher (Horgan, 1999) and 
improve the effectiveness of large class teaching. 
Providing opportunities for active learning while 
relevant in many teaching contexts, is particularly 
relevant in the context of IS education, encouraging 
students to develop the skills, knowledge and 
understanding appropriate to their specialisation, and 
become life-long learners who are able to question, to 
think critically and independently and to learn. 

THE CASE STUDY: 

USING SMALL GROUP PROBLEM- 
SOLVING/DISCUSSION EVENTS TO 
AUGMENT LARGE CLASS TEACHING 

Information Systems (IS) Development is a Level 2 
undergraduate course . This course focuses on the 
analysis, design and implementation of business 
information systems and aims to provide students with 
a “good start in understanding the development and use 
of information in organisations through information 
systems and technology (1ST)”. IS Development is a 
practical, team-based activity. Successful IS develop- 
ment requires appropriate subject and business 
knowledge as well as an active engagement with subject 
and context, and critical thinking that leads to the 
creation (or modification) of information systems. IS 
development, as undertaken by the professional (ie. 
systems analyst), involves analytical, managerial, 
technical and interpersonal skills. Helping students to 
develop the softer skills of IS development (ie. 
managerial, analytical and interpersonal) is an essential 
goal of teaching in this subject. 

The lecture defines the main context of iace-to-face 
teaching in this course accounting for two-thirds of the 
direct teacher-student contact time.' Although the 
lecture is useful for presenting and explaining tiie 
theoretical framework and guidelines for IS 
developm ent, for teaching to be effective, students must 
also engage in learning experiences that enable them to 
apply concept knowledge and develop the ’softer' skills 
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of problem-solving, systems thinking, communication, 
and working with others. Such experiences are not 
easily furthered by the lecture approach as a teaching 
strategy. It is therefore important that opportunities be 
created that support active engagement in critical and 
creative thinking within context and about the business 
problem being addressed. 

In this teaching context the lecture/discussion approach 
is used create opportunities for student engagement and 
to enhance the effectiveness of large class teaching. 
Here the lecturer directs attention to and explains some 
essential information (ie. teaching as transmission) - this 
is followed by questions or problem-solving exercises 
for students to work through individually, in small 
groups, or as a whole group. Problem-solving is an 
essential feature of the lecture/discussion approach as 
used in IS Development, and is an important aspect of 
teaching and of learning (Laurillard, 1997). The 
problem-solving exercises helps to vary the lecture and 
renew student attention while engaging "students in 
thinking about the subject matter in ways that are 
designed to improve their understanding of it" 
(Laurillard, 1 997). Pro blem-so 1 ving requ ires students to 
apply knowledge and principles to new situations as well 
as test and reinforce their understanding of what they 
have learned (ie. at the higher level of critical thinking 
and analysis). Although the lecture/discussion approach 
is not new, its effectiveness can only be determined by 
an empirical evaluation within the teaching context. 

To ensure that students do not feel overwhelmed in the 
large class setting, small groups of four persons (also 
called buzz groups or informal cooperative learning 
groups) work together to master the material presented 
in the lecture and apply concepts (Johnson & Johnson 
1994). It is in this context that students share ideas, 
listen to each other, and critique and reflect on their 
responses. Approximately 3-5 minutes are allocated for 
working through the problem. Initial contributions are 
taken from 2-3 groups selected by the lecturer or by 
colleagues. After the initial contributions, groups 
Joining the discussion are especially urged to comment 
on previous responses or propose alternatives. At the 
end of the discussion, the lecturer summ arises the m ain 
points, clarifies outstanding misconceptions, and 
proposes significant alternatives not previously 
mentioned. Three or more such sessions may be 
incorporated into the 1 10-minute lecture session. 



EVALUATING TEACHING EFFECTIVENESS: 
DATA ANALYSIS AND RESULTS 

A self-administered survey questionnaire was used to 
collect data on the effectiveness of the problem- 
solving/discussion events used in lecture sessions. Of 
the 120 forms distributed (and 106 forms returned), 102 
(85%) were determined useable. The respondents 
included Computer Science (CS) majors and IS m ajors 
(33%), and business-re lated majors with some IS (67% ). 
While 79% of the class had taken foundation studies in 
IS, the remaining students (2 1 %) had taken CS-courses 
only. Of the respondents 64% were male and 36%, 
female; 69% were European/New Zealanders, 1 9% were 
Asian and the remaining 12% were distributed among 
Pacific Islanders, Africans and other ethnicities. Eighty- 
five percent of the respondents expected a B-grade or 
higher. 

The following measures were used to assess the study 
variables: 

• Student perception of teaching effectiveness was 
measured on a 5-point scale ranging from "Very 
Ineffective" to "Very Effective". Respondents 
indicated the extent to which the lecture and small 
group problem-solving/ discussion events were seen 
as effective for their learning. 

• Ranking of Teaching Strategies. Respondents were 
asked to identify and rank the teaching strategies 
they found most effective for their learning.^ A brief 
comment on why this was perceived to be so was 
also given. 

• Motivation to learn. Respondents indicated, on a 5- 
point scale ranging from "No, not at all" to "Yes, 
definitely", the extent to which they felt motivate d to 
learn in this course. 

The results showed that the traditional lecture was the 
preferred teaching strategy for 65% of the respondents, 
while small group problem-solving were preferred by 
3 1 .5% ofthe participants. Seventy-one percent rated the 
lecture as very effective for their learning; 5 5% rated 
small groups as very effective. 

The data analyses were carried out using SPSS 9.0. A 
One-way MANOVA test was used to test for significant 
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differences between responses. A median test and a 
Kruskal-Wallis test were used to test the following 
associations between student perceptions of the 
effectiveness of a teaching strategy and expected grade 
and motivation to learn. 

Hla: There is an association between the lecture and 

motivation to learn. 

Hlb: There is an association between the lecture and 

expected grade. 

H2a: There is an association between small group 

problem-solving and motivation to learn. 

H2b: There is an association between small group 

problem-solving and expected grade. 

The results of a One-Way MANOVA test provided 
evidence of significant differences in student responses 
(F = 5.607; p = 0.02), suggesting that students were 
distinguishing between the teaching strategies. The 
median test provided strong support for accepting 
Hypotheses Hla and H2a: there is an association 
between motivation to learn and small group problem- 
solving (x^ = 20.742, p ^0.001) and lectures 
(X^ = 16.102, p ^ 0.01). The Kruskal-Wallis test 
provided further support for accepting Hypothesis Hla, 
suggestingan associationbetween small group problem- 
solving exercises and motivation to learn (x^ = 14.609, 
p ^ 0.05). Some support was also provided for 
Hypothesis H2b: there is an association between small 
group problem-solving and expected grade (c^=8.149, 
/?£0.05). Hypothesis lb was not supported. 

Respondents were also asked to comment on the 
teaching strategies which they perceived to be most 
effective for their learning. There were 40 comments on 
the effectiveness of the lecture and 19 comments on 
small group problem-solving. Student comments 
provided some support for including small group 
problem-solving in lectures. Participants indicated that 
small group problem-solving ^'gets people thinking"', 
provided opportunities "to apply what we are learning", 
"encourages thought and [is] good for getting different 
viewpoint" , exposed students to "real life examples", 
"made you think about what had just been discussed" , 
and "made understanding the ideas a lot easier by 
seeing it practically applied". Students also commented 
on "collective knowledge" and that "people can share 
each other's ideas [and] get improvement at the same 



time", and have an opportunity to "see different 
viewpoin ts on the same problem" (cooperative learning). 
One respondent also commented that this "relieves the 
boredom of lectures" , 

Consistent with expectations, lectures were preferred for 
coverage and explanation (['Covered a lot of ground and 
explained important po ints in detail"), clarification (e.g., 
"Some aspects explained further than outlined in text" , 
"Makes it more clear - with people explaining rather 
than reading it myself'), providingdirection forlearning 
("Know where you are going"), and stimulating interest 
(Informative and interesting as well as stimulating"), 

DISCUSSION AND IMPLICATIONS 

The results of this study suggested that problem-solving 
in small groups with follow-up discussion was 
particularly effective for providing opportunities for 
students to think, to question, to listen and reflect, to 
apply concepts, and to learn cooperatively from each 
other. Although the lecture and lecture setting have 
limitations, the study did not suggest that active learning 
displace the lecture but rather that active learning 
techniques be used to overcome the limits of the lecture 
and enhance the effectiveness of large class teaching. 
Indeed the results suggested that students were 
distinguishing between the teaching strategies, and that 
the perceived impacts were both different and 
complementary. The small group problem-solving/ 
discussion events in particular, were associated with 
expected grade and motivation to learn. 

In addition to promoting active learning in lectures, the 
problem-solving/discussion events pro vide an invaluable 
opportun ity for the lecturer to reinforce main po ints, to 
find out more about the quality of student learning and 
to provide feedback. If students demonstrated a lack of 
understanding during discussions, this was clarified to 
help the group (andthe class) understand, reflect on, and 
correcttheir responses. It was also important that groups 
felt assured enough to takeon the problem-solving tasks 
without the fear of failure, and that they were willing to 
participate, take risks, and test and share the ideas of the 
group (Davis 1993; Ramsden, 1992). 

Despite limitations (e.g., context, instrumentation) this 
case study provides some evidence to suggestthatactive 
learning supports higher level teaching conceptions and 
goals and can be used to effectively enhance IS teaching 
in the large class setting. Further study is needed to 
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assess the contribution of problem-solving/small group 
discussion and other active learning techniques to 
student learning and learning outcomes. 

END NOTES 

1. Tutorials, laboratory sessions and project work are 
also a part of the teaching/learning context in this course. 

2. Teaching in this course utilised a variety of teaching 
methods which include the project methods, lab 
exercises, and tutorials. For the purposes of this study 
only those rankings and comments related to the lecture 
strategy and the small group discussions are reported. 
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ABSTRACT 

This paper draws on an extensive review of literature associatedwith telecommuting and looks at features that might 
affect the offeringand take-up of distance education, particularly distance education involving computer applications, 
telecommunications and web-based, off-campus delivery of courses or components of courses. The issue is discussed 
from the perspective of the individual, the organisation (the educational institution) and the wider community. The 
aim of the paper is to put forward a possible research model for the evaluation of distance education. 



INTRODUCTION 

The paper begins with a review of the benefits, costs 
and risks associated with telecommuting (Gray et al., 
1993; Turban and Wang, 1 995; Ford and McLaughlin, 
1995; Ellis and Webster, 1997) and suggests that several 
of the matters here have direct relevance to the distance 
education decision. This theme is continued by an 
examination of the enablers, drives and constraints 
(Mokhtarian and Salomon, 1994; Tung and Turban, 
1996) which provides some insight into the factors that 
are likely to influence the acceptance of this form of 
education. 

Parallels aredrawnbetween educational andwork-place 
tasks the understanding of prescribed material, 
assignments, experiences and acquisition of knowledge 
on one hand and the components of a job on the other. 
Using a theoretical task model to encompass the 
component, co-ordinative and dynamic themes of 
complexity (Wood, 1986), the task characteristics of 
uncertainty and equivocality (Daft and Macintosh, 1981) 
and the organisational issues ofresourcesand scheduling 
of work (Thompson, 1967), a set of attributes for 
educational tasks is developed. It is proposed that this 
model form a central component of a research model for 
the evaluation of the suitability of educational tasks to 
distance education. 



In respect of the personal attributes of the individual, 
there are parallels with the telecommuting literature too. 
These are most likely to be in the areas of personal 
characteristics such as the ability to get information 
required, knowing when advice is needed, the ability to 
solve one’s own problems and good self-management 
(Venkatesh and Vitalari, 1992; Gray et al., 1993; 
Wheeler and Zackin, 1994; Mokhtarian and Salomon, 
1 996a) and the home environment (Yap and Tng, 1990; 
Mannering and M okhtarian, 1995). 

To some extent the role of the academic is analogous 
with that of the supervisor. As the supervisor controls 
allocation, timing and resources for tasks (Starr, 1971), 
the academic controls task content, timing and the 
required resources. Accordingly^ this literature, as it 
relates to telecommuting, offers some assistance in 
studies of distance education. 

Telecommuting literature also provides some poin tersto 
demographic influences on the preference to 
telecommute age, gender, time in the work-place, job 
type, education, transport, presence of small children and 
the number of cars in the household (Mokhtarian and 
Salomon, 1997; Belanger, 1999; Dickand Duncanson, 
1999) some of these seem to have relevance to the 
decision to engage in distance education. 
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ADVANTAGES AND DISADVANTAGES 



,,,and the disadvantages. 



For the Individual 

The telecommuting literature (Olson, 1983; Rice, 1987; 
Ford and Butts, 1991; Gray et al., 1993; Mokhtarian 
and Salomon, 1994; McQuarrie, 1994; Turban and 
Wang, 1995) suggests the following as potential 
advantages and disadvantages of telecommuting each 

item is discussed with a view to its applicability to the ^ 
student undertaking distance education, using som e form 
of information and communications technology to aid 
him in the associated tasks. First, the advantages... 

Reasons associated with travel to work such as 
reduction in commuting stress, saving money and 
time andhelping the environment somerelevance 
to distance education; not having to attend on a 
regularbasis may reduce travel costs for the student, 
particularly if long distance travel is involved. In 
this context it should be noted that reduction of 
living costs maybe a significant factor for the 
potential student. Also, this area might be 
broadened to include those for whom travel would 
be impossible, such as those living abroad or in 
remote areas. 

Better able to manage one’s own affairs eg. more 
independence, flexibility, control of the physical 
working environment, to study or pursue personal 
interests particular relevance perhaps to the post- 
graduate student in the sense of better managing 
work commitments. 

To be able to work if sick, disabled or look after a 
sick child or other dependent potentially 
increases the possibility of education for those who 
may be disabled or extensively involved in the care 
of dependent children or other relatives. 

To reduce the stress experienced in the office 
relevantperhaps to those who might find the campus 
environment threatening or intimidating. 

To spend more time with one’s family covered 
above. 

To get more work done campus life offers many 
distractions for the student; while mostly seen as an 
advantage, some students may benefit from the 
possibility of removing themselves from these 
distractions. 



More difficult to work at home due to less help 
available, motivational problems, increased family 
conflict and distractions one might expect these 
to be serious impediments to distance education for 
many people, requiring particularpersonal attributes 
for them to be overcome. 

Viewed negatively by management, being “out of 
sight and out of mind if we interpret 

“management to mean academic staff, there could 
well be a feeling among distance students that those 
with physical access to the academic staff get 
enhanced help and assistance. 

Exploitation by management missing out on 
overtime or having to w ork extra tim e to cover peak 
periods “management in this sense could be 
interpreted as the university administration which 
supplies resources and occasional casual work to 
supplement student incomes. 

Travel time can be used productively, to run errands, 
or provides a break between home and the office 
travel is seen as a time for completing assignments, 
reading, study, etc. 

The office is nicer/better equipped than a home 
office would be a significant issue for potential 
distance students may be the need to equip a home 
study area with a PC and appropriate software, 
telephone line, communications software. 

The social interaction found at the conventional 
workplace missing out on the extra-curricular 
activities that take place on campus could be viewed 
by many as a serious impediment to distance 
education. 

The professional interaction found at the 
conventional workplace getting to know one’s 
fellow students, easy access (formal and informal) to 
academic staff. At a more strategic level, a 
diminished educational experience may result. 

For the Organisation 

Advantages to the organisation from telecommuting 

normally centre around productivity, better use of an 

employee’s time, a wider pool of recruits on which to 

draw, saving on conventional office space and an 
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extension of working hours (Katz, 1987; DuBrin, 1991; 
Grayetal., 1993; H amb len, 1999) . Similar advantages 
could accrue to universities offering distance education 
increasing staff work loads and the ability to offer 
courses outside conventional hours may appeal to 
university administrations; the wider pool ofstudents is 
already seen as a major driving force and there could be 
savings in lecture halls, tutorial rooms, laboratories and 
other on-campus facilities. 

Disadvantages from telecommuting are to do with 
changing the way organisations work and function, 
duplicating equipment costs, absence of key personnel 
from the conventional workplace, morale problems and 
security (Ford and Butts, 1991 ; Filipczak, 1992; Li and 
Gillespie, 1994; Tamratetal., 1996; Orlikowski, 1996; 
Dick and Duncanson, 1999). Re-skilling academic staff 
and changes to more traditional ways of teaching may 
present problems, there will be additional costs in 
supporting'students’ online access and the absence of 
students from the conventional classroom may diminish 
the research and teaching standing of the university. 

For the Community 

Potential reduction in the demand for transport 
infrastructure, reduction in pollution and benefits to local 
or rural communities are often cited as possible effects of 
telecommuting (Blanc, 1988; Mokhtarian et al., 1995; 
Nilles, 1996). There is somescope for these benefits from 
distance education, perhapsthemost significantmay be the 
reduction in the subsidies required for public transport. 



Against this, business activity in the city centres and 
university towns may fall, travel may increase in 
outlyingareas and energy consumption in thehome may 
rise (Gray et al., 1993). 

ENABLERS 

There is along list ofelectronic enablers which facilitate 
telecommuting PCs and laptops, printers, modems, 
copiers, fax machines, cellular telephones, answering 
machines, high speed communications links and access 
to e-mail and the Internet (Hotch, 1993; Tung and 
Turban, 1996). While clearly not all are required for all 
tasks, this list is a useful starting point for the types of 
electronic assistance that would facilitate distance 
education. At present much of this equipment is made 
available free of charge to students in the traditional 
campus environment considerab le expense would be 
incurred by the student in equipping himself with such 
technology. On the other hand many universities are 
moving to requiring(or expectin g) students to have such 
technology available at home. 

TASK SUITABILITY 

The following model, constructed from the 
telecommuting and task literature, outlines various 
aspects of task properties that make task suitable for 
telecommuting. 



FIGURE 1 
A TASK MODEL 
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Based on original models of task suitability for 
telecommuting (Olson, 19 83; Huws et al., 1990) the 
model suggests support for these properties from the task 
related literature. 

The original telecommuting models suggested that tasks 
may lend themselves to telecommuting if physical 
requirements (for resources and equipment) are kept to 
a minimum, the staff member is in a position to control 
the pace of his work, the work has defined deliverables, 
required concentration, has specific milestones set and 
has minimal need for communications with one’s 
supervisor or fellow employees. There are obvious 
parallels here to those tasks whic h are likely to be part of 
distance education assignment writing, research, 
understanding course notes and lecture material and 
examination preparation. 

In terms of task complexity (Wood, 1986), in general 
terms as the degree ofcomplexity rises, the task becomes 
less suitable (or more difficult) for telecommuting. The 
same may be said of distance education. Component 
complexity is a function of the number of distinct acts 
that are required to perform the task and the number of 
information cues to be processed in performing these 
acts. Comp onent com plexity is also affected by the task 
being dependent on completion of other tasks. The type 
of task may have relevance here too some concepts 
may be difficult to explain or demonstrate without 
“hands on experience for example dissection, 
modelling and instrument operation. Co-ordinative 
complexity refers to the form and strength of 
relationships and the sequence of inputs. Wood suggests 
that the more complex the timing, frequency, intensity 
and location requirements, the greater the knowledge and 
skill the individual must have to be able to perform the 
task. Changes in the acts and information required or in 
the relationships between inputs and products Wood 
calls dynamic complexity. This too can create shifts in 
the knowledge or skills required. 

To illustrate, if we consider component complexity, 
tasks with minimal component complexity may be those 
such as reading a study guide, notes ora text book and 
answering a series of “review questions. At the other 
end of the scale, research using multiple resources, 
including hard copy and electronic journals, text books 
and the Internet, discussions with a colleague and 
writing up a summary of the research may present 
difficulties for the distance education student. Likewise 
co-ordinative complexity could range from one person 
completing an assignm ent to working as part of a team. 



with each member responsible for various components 
and then the team having to link them together to 
produce a final product. 

The task characteristics (Daft and Macintosh, 1981; 
Daft et al., 1987) of equivocality (ambiguous meanings 
or instructions) and uncertainty (about what is required 
or how to go about it) are relevant to tasks involved in 
distance education too considerable difficulty might 
be expected to be experienced by the student if tasks are 
not clearly explained with no ambiguity and specifiedto 
reduce uncertainty. 

Similarly, the environment in whichthe tasks take place 
(Thompson, 1967) may have some relevance to their 
suitability serial depen dence refe rs to the need to wait 
on others (academic or student) in order to commence or 
complete one’s own work . Also relevant is the degree 
of“rietworking andteam buildingthateducational tasks 
are designed to include. 

PERSONAL ATTRIBUTES 

Successful telecommuters display certain traits(Gray et 
al., 1993). According to the authors, these traits are: 

the ability to make sound decisions, know where to 
get the information that leads to that decision 
making process or completion ofthe task and know 
when they need advice; 

the ability to solve their own problems this might 
require a knowledge ofthe organisation, the tasks or 
the technology and an analytical approach to 
problem solving; and 

good self management self motivation, time 
management, the ability to assess their own work 
and to be able to put these skills together to deliver 
quality work on time. 

There is considerable support for these traits in the 
literature see also (Venkatesh and Vitalari, 1986; 
Mokhtarian and Salomon, 1996b; Lewis, 1998). 
Confidence in working within the e lectronic community 
(Venkatesh and Vitalari, 1986; Hesse and Grantham, 
1991; Trevino and Webster, 1992) may also be an 
important attribute the telecommuter is isolated from 
“help , and runs the risk of being seen as incapable of 
working with the required tools, or being seen as 
incompetent if he/she does not hold the necessary skills. 
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Other aspects identifiedin various studies by Mokhtarian 
and others includethe need for self discipline, househo Id 
interaction problems and aversion to risk (Mokhtarian 
and Salomon, 1994), susceptibility and aversion to stress 
(Mokhtarian and Salomon, 1997), (see also (Trentet al., 
1994)) and the desire to get more work done 
(Mokhtarian et al., 1994). 

For the distance student, knowing where to get relevant 
information and when to seek advice would seem to 
have particular importance, as does the ability to solve 
his own problems the added reliance on information 
technology and communications equipment gives this 
aspect added weight. Under-graduates are more likely 
(perhaps than there post-graduate counterparts) to have 
motivational problems and will need to develop time 
management skills to enable work of an appropriate 
quality to be delivered on time. On the subject of the 
household environment, the telecommuting issues 
(Mannering and Mokhtarian, 1995) ofpresence of small 
children, number of people in the household and family 
orientationmay also have some effect on the preference 
to study at a distance. 

THE SUPERVISOR 

The supervisor has a enigmatic role in telecommuting 
on one hand, without the supervisor’s approval of 



individual instances, telecommuting is unlikely to take 
place while on the other, the attitude of the supervisor 
does not seem to affect the preference to telecommute 
(Dick, 2000). Nevertheless, if we align the role of the 
supervisor with that of the academic, some issues do 
arise. It has already been noted that one of the 
disadvantages of telecommuting to the organisation 
relates to changes to the way of working. Academic 
staff will need to learn new skills, particularly IT related 
ones. They will need to be prepared to “formalise 
presentations and the learning experience imparted to a 
considerable degree and to correspond with students by 
the, perhaps unfamiliar media of e-mail, “chat rooms 
and “bulletin boards. These changes will not be easy 
and are likely to involve universities in considerable 
upheaval. 

A RESEARCH MODEL 

The above suggests that the following model may be 
useful in the evaluation of distance education as an 
alternative for students and universities. 



FIGURE 2 

A RESEARCH MODEL 
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CONCLUSION 

Many education institutions are considering, or have 
already implemented, distance education programmes. 
Research on these programmes is just beginning. There 
is a need for the evaluation of such programmes against 
a sound research model. The essential contribution of 
this paper is to bring the telecomm uting related literature 
together in a manner which allows the development of a 
preliminary research model for studies of the provision 
of distance education from the educational institution 
perspective and for the desire to partake in distance 
education from an individual perspective. The model 
brings together the tasks (and the related issues of 
schedulingand control ofwork), personalcharacteristics 
of the individuals, the perceived advantages and 
disadvantages of undertaking a course of study in this 
way and the necessary underlying technology. 
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ABSTRACT 

This paper describes some of the major policy issues that occurred during the last fifteen years that were faced by 
the university and IT management of a regional Australian university. The changing context of the Australian higher 
education industry during this time is described. An unusual feature of this period described in the paper, is a 
collaborativesystems development project in which most Australian universities participated, but which did notmeet 
its full objectives . Towards the end of this period, the software market for university administration systems became 
more active, which lead to the adoption of Enterprise Resource Planning systems by this and other Australian 
universities. 



INTRODUCTION 

Information Technology (IT) occupies a high profile 
within Australian Universities. Universities were among 
the first organisations to acquire computers because of 
their usefulness in solving mathematical problems. 
Today computers are used in teaching and research in 
virtually every discipline. The Internet which originated 
in response to the needs of researchers now pervades 
many university activities including advertising and 
promotion and disseminating teaching m aterials. As well 
as being essential to the ’core business' of universities, 
that is teaching and research, IT has an im portant role in 
university administration underpinning the so-called 
'back-office’ systems that support these core activities. 
Major university administrative applications include 
Student Administration, Human Resource Management 
and Finance. 



This paper examines some major issues that have arisen 
over the last fifteen years and the industrial context in 
which they are situated, from the perspective of a 
regional Australianuniversity . Initially somebackground 
information on the industrial context is presented. 

THE AUSTRALIAN UNIVERSITY SYSTEM 

The majority of Australia’s 37 universities are publicly 
funded (only two universities. Bond and the Australian 
Catholic University are private organisations). Currently 
(August 2000) they fall within the responsibility of the 
Federal Department ofEmployment Training and Youth 
Affairs (DETYA). The Australian University system is 
characterised by diversity, even though since 1987 a 
Unified National System (UNS) introducedby Minister 
Dawkins, has been in effect The mains effect of the 
UNS was to re-designate a number of non-university 
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institutions such as Colleges of Advanced Education, 
which were m ore vocatio nally orientated as universities, 
along with the traditional academically strong and 
research orientated 'pre-1987' universities and to 
promote larger more viable institutions via mergers. 
Raising the overall number of universities increased the 
potential for collaborative activities. Another 
consequence of the 1987 reform was that in all of the 
newer universities, research which had formerly been a 
low key activity was placed higher on the agenda with 
the concomitant refocussing of budgets and institutional 
effort. 

Increased Competition 

During the 1990’s misgivings arose in government 
circles about the overall size of the higher education 
budget. “The link between the public investment and 
return on this investment in terms of visible national 
revenue has not always been clear and obvious to the 
tax-payers, or for that matter to the Government 
(Hamilton 1 997). In an era of econom ic rationalism there 
is less inclination to pursue public good that results in 
private benefit. Commenting on the West Higher 
Education Review, committee member and Central 
Queensland University (CQU) Vice Chancellor and 
President Lauchlan Chipman commented. “One of the 
most difficult questions ... is whether the benefits of 
higher education are primarily private benefits - benefits 
to the individual graduate - or primarily public benefits - 
benefits to society at large. The case for public subsidy 
is based on the idea that there is a significant social 
benefit. The case for requiring the student to contribute 
is based on the idea that a degree brings significant 
personal advantages, especially in job opportunities, total 
lifetimeearnings, geographical mobility, and social class 
(’status') mobility (Chipman 1997). These attitudes have 
coincided with a decrease in the level of government 
support for publicly funded universities. “Where once 
the Government provided about 90% of university 
funding in 1983 by 1997 this had declined to about 
57% (Hamilton 1997). Surprisingly this decline in 
government funding was accomplished at a time when 
there was a considerable increase in the number of 
students enrolled. Some of this increase came from full 
fee paying overseas students, which was one way in 
which universities responded to the contraction in 
government support. Increased competition forstudents 
tended to create a more competitive atmosphere in the 
Australian University industry during the period under 
examination. 



The biggest challenge for the AV-CC is that as vice- 
chancellors we agree on less and less. Each of us has a 
duty to do what we can to advance the interests of our 
own university, and each of us has a national duty to 
contribute our expertise and experience to shaping policy 
for the higher education sector in Australia as a whole. 
(Chipman 1999b) 



This change in market focus created additional demands 
for management information. 

CASMAC 

Public funding remains important to Australian 
universities, accounting for at least 50% of revenues in 
many cases. Public funding implies public 
accountability, which means that universities must 
provide various statistics to DETYA to enable a degree 
of public monitoring. The gathering of statisticsis a type 
of information request to which un iversities would like 
to be ableto respond expeditiously through the use ofIT. 
The fact that many university student record systems 
were developed during the 1970s using 3*^^ generation 
software technology meant that for many universities it 
was a struggle to keep pace with chan ging requ iremen ts 
for information from government. A characteristic of IT 
systems of this era (ie the 1970’s) is that they are 
relatively inflexible and do not lend themselves easily to 
changed requirements. Difficulties in meeting 
information requests from government lead universities 
to consider a cooperative approach to IT systems 
developm ent. 

The Core Australian Specification for Management 
and Administrative Computing (CASMAC) steering 
committee was formed in 1989 by the Australian Vice- 
Chancellors’ Committee (AVCC), the council of 
Australia’s university presidents. The AVCC is the peak 
organisation representing Australian universities 
nationally and internationally and seeks to advance 
higher education through voluntary, cooperative and 
coordinated action. 

CASMAC is based on a similar project initiated in 1 988 
by the University Grants Committee (UGC) in the 
United Kingdom that became known as the Management 
and Administrative Computing(MAC) initiative. These 
projects were undertaken because of the difficulties 
being experienced by universities in maintaining and 
redeveloping administrative systems in order to satisfy 
increasing demands for information from government. 
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As the demands for information came from the 
government the same demands were placed on all 
universities. Both CASMAC andtheUKMAC initiative 
were based on the premise that there was a high degree 
of commonality in the core functionality needed to 
support the administrativeand management functions of 
universities. The AVCC believed that there werebenefits 
to be obtained by specifying these requirements and 
sharing the development and ongoing costs of the 
resulting systems. 

The Steering Committee was successful in its bid to the 
Department of Employment, Education and Training 
(DEET) for funds to undertake a feasibility study to 
determine if a similar approach to MAC could be 
applicable in Australia. As a result of the feasibility 
study, it was agreed to proceed with the preparation of 
the Core Australian Specification for Management and 
Administrative Computing. Between April 1991 and 
July 1991 a small technical specification team together 
with consultants from Price Waterhouse used the MAC 
blueprint as a base to develop CASMACo. From 
December 199 1 to May 1992 progressive revisions were 
made to the specifications. CASMAC specifications use 
the structured system specification techniques of data 
flow diagrams, data model diagrams and a data 
dictionary. It has been suggested that the specification 
may cover 80% of each university's functional 
requirements but this is obviously a very rough 
assessmentof something that cannot really be quantified. 



CASMAC is intended to meet the following four 
objectives: 

Objective 1: 

Specify and maintain the core functional 
requirements necessary to support the business and 
management functions of Australian universities. 

Objective 2: 

encourage and provide the basis for cost effective 
collaborative development and acquisition of 
adaptable and integrated core administrative 
computing applications which will: 

1. meet Australian universities' core operational and 
information requirements; 

2. provide the means of complying with statutory 
responsibilities and managerial obligations in a 
timely manner; and 

3. facilitate the effective deployment of university 
resources. 



Objective 3: 

maximise the ability to respond to external reporting 
requirements, particularly those of go vernm ent, in a 
cost effective and timely manner. 

Objective 4: 

allow for the flexible adoption and/orexploitation of 
new and evolving technology and facilitate the 
implementation of innovative approaches for 
management and administrative systems. 

The national Core Australian Specification for 
Management and Adm inistrati ve C ompu ting defines the 
core functional requirements of the management and 
administrative needs of Australian universities for: 

Student Systems 
Human Resources 
Finance Systems 
Physical Resources 
Research and Consultancy 
Executive Reporting 
(AVCC 1997) 



While the term ‘CASMAC’ technically refersto only the 
core specification it is often used to include those 
dimensions of the project that emerged subsequently. 

In July 1 992 a Request forinformation (RFI) was issued 
to potential suppliers of systems which would meet the 
requirements specified in CASMAC. This RFI required 
suppliers to indicate how they might deliver CASMAC- 
compliant applications to universities and to address a 
range of specified issues. Nineteen responses were 
received. A Request for Tender (RFT) was issued to five 
potential suppliers shortlisted from the evaluation of the 
RFI. The responses received were evaluated against the 
following criteria: CASMAC compliance, cost, 

company/consortium profile, technology platform, time 
frame and delivery schedule, control/ownership/support 
and adaptability and customisation capability. 



Between January 1993 and March 1993 universities 
were invited and encouraged to commit to a 
collaborative group project based on one ofthe suppliers 
responding to the RFT. Such a commitment would 
involve: 

agreement to meet a share of the total cost of 
achieving the CASMAC objective through a joint 
venture with that supplier; 
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representation on the Management Committee 
established to oversee the project; 
commitment to the provision of resources in the 
form of staff time and expertise for systems 
development, testing and implementation; and 
no commitment for an institution to take any of the 
applications with timing of implementation at the 
discretion of the institutions. 

(AVCC 1996) 



In April 1993 based on the responses to this invitation 
the Steering Committee recommended to the AVCC that 
an agreement be signed with two selected suppliers, 
Oracle and Coulson Heron Associates (CHA), and that 
Management Committees for each projectbe established, 
reporting to the CASMAC Steering Committee. During 
the following weeks, 19 universities committed to 
forming a consortium based on the CHA proposal using 
the PowerHouse fourth generation development 
language. This consortium was subsequently called 
UniPower. 



The UniPower consortium has been the firstto announce 
details of the system it is developing. On the 11 ^^ July 
1993 it signed a $10.5 million deal with British software 
house Coulson, Heron and Associates (CHA) and 
Canadian-Us software house Cognos to begin 
development of the system based on the software 
developed by CHA in the UK. The $10.5 million 
software cost will be spread over the 19 universities with 
contributions being determined by the relative size of 
each. (The contribution of CQU being $185,683). 
(CASMAC Bulletin #1, 1994) 



Of these 19 universities, 16 were already using Cognos 
(Powerhouse) products. 



For UCQ' the decision to join the Powerhouse family 
was easier as there was already a strong Powerhouse 
presence with the existing Student Record application 
being written in the Powerhouse 4GL language. 
(CASMAC Bulletin #1, 1994) 



In the same period 1 1 other universities committed to 
forming a consortium which would use the Oracle 
relational database management system. This consortium 
was subsequently calledtheUniOnGroup. Significantly, 
both successful tenderers were also successful in the UK 
tender process and, therefore, had already developed a 
set of systems that could be used as the basis for the 
Australian systems. Neither group appears to be 



particularly biased with respect to the inclusion or 
exclusion of 'pre-1987' universities. 

Later in 1 993, three oftheremaininguniversities, known 
as the Natural Group, agreed on a Memorandum of 
Understandingforthe joint development, acquisition and 
exchange of CASMAC compliant software based on the 
Natural fourth generation language. The four other 
universities either determined to develop their own 
software in-house or to acquire software from other 
vendors. DEBT allocated $2.7 million to the UniPower 
consortium in 1993. This project eventually ran into 
problems and no useful systems were developed. Legal 
proceedings were instigated by the consortium in 1997 
to seek recompense for this failed investment. 



For many IT investment decisions in universities, the 
question is not ’should we invest' but 'where and how to 
invest'. It is often not a choice of whether to develop, 
upgrade, or replace a system, but which system will 
efficiently deliver the required benefits, and how should 
it be obtained and installed. For such a system, which 
could be termed an operational or competitive necessity, 
’return on investment’ is not usually a very relevant 
measure. Where the system is fundamental to running 
the organisation anddelivering services, there may be no 
choice but to invest. In many cases, and especially in the 
public sector, it is difficult to quantify all of the costs 
and benefits (Stacey 1993). This may cause some 
universities to neglect the cost-benefit analysis of 
proposed IT projects. Some of these will be operational 
or competitive necessities, which will not give you any 
competitive advantage, but will keep you in the game 
and thus are necessary for survival. 

Many university administrative IT projects would fall 
into this category. In attempting to address the need for 
such systems, most universities have collaborated in the 
CASMAC initiative. Collaborative development has 
been proposed as a way to share the costs of developing 
standardisedIT systemsfor areas such as finance, human 
resources, student administration and management 
information. 

The objectives for CASMAC include cost-effective 
collaborative development and acquisition of 
administrative systems to meet the operational and 
information requirements of the universities, and 
especially to enable a standardised response to the 
universities' statutory requirements and external 
reporting responsibilities to government. 
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The collaborative focus of C ASM AC was therefore 
intended to provide a framework for institutional 
developm ent, towards which universities could m igrate 
in the knowledge that systems obtained from other 
universities, or possibly developed in conjunction with 
other universities, would have minimum integration 
problems. 

In addition, this separation of the project from individual 
universities seems to have led to a distancing from, and 
lackof ownership by, university line management, even 
though the development process involved considerable 
consultation, and software development teams usually 
included application specialists from within the 
universities. The interviewers often heard both university 
IT managers an d senior admin istrators say they were not 
focused on the administrative systems of the institution 
because they were 'waiting for CASMAC. It did not 
have senior management attention, although some did 
express concern about how long it was taking to get 
UniOn or UniPower systems in place. These delays 
meant that 'interim systems' were sometimes being 
developed or purchasedby some universities to meet the 
immediate needs of the institution. 

Competition between universities has grown since 
CASMAC began in 1991, and this is leading to 
increasing strategic differentiation. Perhaps this 
changing strategic focus within universities has 
contributed to the lack of attention to and 'ownership'of 
the CASMAC initiative by individual universities. 
(Vitale et al 1997) 



INFORMATION SYSTEM DEFICIENCIES 

CQU decided that their Finance and HR systems would 
need replacingbefore a solution would become available 
from UniPower. Consequently interim stand-alone 
systems for Finance and HR were purchased and 
installed in 1991 and 1993 respectively. 

Changing reporting requirements for government, new 
management requirements for information because of a 
changing student profile, including full fee paying 
overseas students, an increase in research activity plus an 
awareness of newer and superior IT appearing in the 
market highlighted the inadequacies of current IT 
systems. 



In 1997 CQU was faced with a number of issues. The 
existing student record system had been designed on a 
model that had a single campus, recognised two 
semesters, was not Year 2000 compliant and used an 
inappropriate database structure. A decision by the 
University to operate over four terms rendered the 
existing system inoperable. There were no commercial 
systems in the market that would solve this problem in 
the required time fram e. An in-house project was 
initiated to amend the existing system to cater for four 
terms, to make it Year 2000 compliant, to change the 
database platform, and to cleanse the data. The structure 
and processes used by this system were largely 
undocumented and posed a potential risk to the 
University. It was reco gnised th at considerable resources 
would be required to address these issues and that the 
potential returns would be minimal. 

The Vice-President(Administration)calledall interested 
parties to a meeting to review the student system on 7^*^ 
August 1998. That meeting identified a number of 
outstanding issues in relation to the system. It was also 
recognisedthat many of the problems stemmed from the 
outdated and often redundant processes that are 
associated with the system. This issue along with a 
proposed review of IT provision and services was 
referred to the monthly meetings of the University's 
Senior Management Group. The group met on two 
occasions to address the matter. Following further 
discussions with the Associate Deans of Teaching and 
Learning it was agreed that whilst a University-wide 
review of IT was important the greater need was to 
improve the University's administrative processes. The 
Director Information Technology Division (ITD) was 
then requested to develop an appropriate RFP. 

The RFP was developed as a result of a series of 
meetings and discussions amongst ITD, the 
administrative functional areas, the Associate Deans 
Teaching and Learning and key faculty based users. The 
RFP had a primary focus o Student Administration but 
also addressed the University's long-standing 
requirement for integrated systems and in addition 
requested proposals for Financials and Human Resource 
systems. The RFP was released on 7^’’ December 1998 
and sent to the three potential providers of Student 
Systems in Australia. These providers were PeopleSoft 
Australia, Deakin Software Services and Technology 
One. 

(Central Queensland University Integrated Systems 
Project Final Report May 1999) 
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Two of the three potential providers, Deakin Software 
Services and Technology One were Australian software 
houses offering systems that had originated from the 
CASMAC project (see Figure 1). PeopleSoft are an 
American company with an international presence in the 
Enterprise Resource Planning (ERP) system market. 
Their ERP product is primarily recognised as having a 
strong HRM capability, however they have developed 
Student systems for universities from an American base 
which have been adapted to Australian an New Zealand 
requirements. Following a process that evaluated the 
responses from these providers, PeopleSoft Australia 
was selected as the preferred software supplier for CQU 
on 12 March 1999. Eleven Australian universities are 
reported as having signed co ntracts to adopt PeopleSoft 
systems (The Australian April 26 2 000). 

The CQU RFP did not express CASMAC compliance as 
a general requirement, opting instead for: 



Complies with Federal and State legislation. Also 
enables the University to satisfy the reporting 
requirements of government bodies such as DETYA, 
ATO, and ABS and other external bodies. 

(CQU Request For Proposal 7^*’ December 1998) 



The interest the University had in more efficient 
administrative processes is evident in: 



Facilitates Business Process Re-engineering, including 
Work Flow automation as a design concept across all 
modules. 

(CQU Request For Proposal December 1998) 



and reflecting the limitations of existing systems and 
hopes for the future in: 



Rapid changes in the higher education environment 
make it imperative that any administrative computing 
systemsput into place do not constrain future courses of 
action. 

(CQU Request For Proposal December 1998) 



Even if a full cost benefit analysis of the impact of 
installing new IT systems has not been performed, 
possibly because of the complexity of the calculation, 
there seems to be an implicit assumption that savings 
will eventuate. 



Many of the savings that are part of the pay-off of new 
systemsarise because their imminent arrival disciplines 
us to review the processes and transactions through 
which the organisation conducts its internal and external 
business. (These are reviewswe shouldbe conductingin 
any case.) We must conform our processes to the 
systems we are purchasing, and not the other way round. 
(Chipman 1999a). 



Figure 1 . System Development From CASMAC 
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These savings are not achieved directly through the IT 
budget, by replacing ageing costly to maintain systems 
with newer systems requiring less maintenance 
expenditure but from revised business processes. The 
introductionofthe new integrated IT systems createsthe 
opportun ity to introduce new business processes that are 
more efficient than their predecessors. Projects like 
PeopleSoft project at CQU are not simply IT projects 
designed to produce better information for managers and 
government but are also instruments of business change 
and institutional reorganisation and restmcturing.In this 
sense the current PeopleSoft project has a different 
emphasis and rationale from the earlier CASMAC 
project. 

QUESTIONS FOR DISCUSSION 

1. What are the advantages and disadvantages of 
collaborative IT projects in a particular industry? 

2. What are the implications of this account to theories 
of strategic information systems development for 
competitive advantage? 

3. What appeared to be the determining factor in the 
decision regarding which consortium to join? 

4. How do you account for the comparative success of 
the PeopleSoft product in Australia? What impact 
did the collaborative CASMAC project on have on 
individual IT managers in universities? 

5. Do you view the CASMAC project as a positive or 
a negative experience for Australian universities? 

6. What alternative outcomes may have emerged had 
the CASMAC initiative not occurred? 

7. Do you think the basis for making a decision to 
invest in IT systems is different for public and 
private organisations? Is there evidence suggesting 
the application of more formal investment decision 
making in the commercial sector? Do youagree that 
the necessity to invest in an IT project can be 
assessed in the absence of financial criteria? 



8. What criteria would you apply to the selection 
process at CQU when it needed to choose between 
the respective products from PeopleSoft, Deakin 
Software Services and Technology One? 

9. How would you define the role of an IT department 
when all of the significant administrative systems 
operated by the organization are packaged software 
products? 

ACKNOWLEDGMENT 

The assistance of Ian Jenkins, Director of the 

Information Technology Division, Central Queensland 

University is gratefully acknowledged for advice and 

assistance and material for this Case Study. 

ENDNOTE 
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ABSTRACT 

Nowhere in the applied business disciplines is the need for continuous monitoring and improvement of university 
degree programs more paramount than in information systems (IS). IS programs, by their very nature, must 
constantly evolve if they are to maintain a reasonable level of currency with rapidly changing state-of-the-art 
technologies and industry practices that leverage them. The importance of assessing the quality of such programs 
is underscored by the fact that performance assessment is becoming an increasingly important issue in accreditation 
decisions. Ideally, such assessments help universities evaluate the quality of the services they provide to their various 
constituencies and provide a mechanism for identifying strengths and weaknesses of their programs. In this paper 
we describe a methodology that is being used to assess strengths and weaknesses within an information systems 
degree program. The identification of these strengths and weaknesses is a preliminary step in the development of 
a larger and more encompassing performance assessment model. 



INTRODUCTION 

The idea of formal “performance assessment” processes 
in higher education is relatively new. Although it has 
not always beenthe case,AACSB standards nowspecify 
thatdegree programs should be systematically monitored 
to assess effectiveness and periodically revisedto reflect 
new objectives and improvements based on 
contemporary theory and practice. To satisfy this 
standard, a school must demonstrate that processes exist 
for “planning, monitoring and revising”curriculum, and 
that these processes do indeed result in new or revised 
curriculums. AACSB guidelines further suggest that 
these processes should include an analysis of the 
educational outcomes as viewed by various stakeholder 



groups. Despite this accreditation requirement, few 
published reports of program assessment processes for 
business programs in general (and information systems 
programs specifically) can be found in the literature. 

A review of the IS literature suggests that a number of 
researchers have examined the needs of employers of IS 
graduates from the standpoint of identifying the 
knowledge and skill sets most often demanded in the 
marketplace (Rogers et al., 1999; Athey and Plotnicki, 

1 992; Jacobson and Armstrong, 1996; Todd et al., 1995; 
Wilson, 1996; Case et al, 1997; Cheney and Lyons, 
1980; Harris & Harris, 1997; Kahn and Kukalis, 1990; 
Leitheiser, 1992; Nelson, 1991; Padgett et al., 1991; 
Young and Lee, 1997). Similarly, IS researchers have 
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studied the knowledge and skills deemed most useful 
and/or important by graduates of various IS programs 
who are now employed as IS professionals (Doke and 
Williams, 1 999; Womble, 1994). This body of research 
provides some important insight into the views of the 
employer and alumni stakeholder groups with regard to 
the currency of IS curri culum s. Less attention, howev er, 
has been given to the perceptions of students nearing 
graduation, and in particular, if and how their views of 
the value of their education (and the associated strengths 
and weaknesses of their degree program) change over 
time as they become an integral part of the IT workforce. 

In a recent search of business and education citation 
databases, only a few articles could be clearly identified 
as addressing performance assessment in higher 
education at the institutional and/or “degree program” 
levels (Haksever and Muragishi, 1998; Johnes, 1996, 
1 998; Hager and B utler, 1996), and only o ne specifically 
addressing performance assessment of IS programs 
(Weistroffer & Gasen, 1995). While our literature 
search cannot be considered fully exhaustive or 
definitive, there is clearly a paucity of research 
addressing these issues. 

PRELIMINARY PERFORMANCE 
ASSESSMENT ISSUES 

The development of meaningful assessment indicators 
for IS degree programs must take into account the 
perceptions of various constituent groups, as well as 
“hard data” related to various inputs and outputs. 
Several constituencies of IS degree programs can easily 
be identified, those being students, faculty, graduates, 
and employers of those graduates. Because of their 
involvement in and knowledge of information systems 
programs and the industry as a whole, these groups are 
in a position to provide valuable insight regarding the 
perceived quality of IS degree programs. The important 
questions for the current study revolve around the 
identification of key strengths and weaknesses of an IS 
degree program. 

Previous research in IS education has focused on the 
identification of knowledge and skill sets desired by 
those that hire IS graduates and related curriculum 
development efforts. The majority of these studies have 
been motivated by the needto steer IS degree programs 
to properly match the ever-changing requirements ofthe 
IS industry, and thus have focused on the perspective of 
the employer constituency. These studies provide 
assistance in identifying what are considered to be 



important core competencies of IS graduates — the 
“outputs” of IS degree programs — as seen by this 
important, external constituent group. From the 
standpoint of program assessment, however, an even 
more basic question is how well an IS program is 
meeting the needs dictated by these core competencies. 
In other words, a preliminary step in the performance 
assessmentprocess is determining what key stakeholders 
(students, alumni, employers and faculty) see as the 
strengths and weaknesses of a given degree program. A 
relatedand interesting question is whetherperceptionsof 
strengths and weaknesses are consistent across the 
various stakeholder groupsand within groups over time. 

The strengths and weaknesses of any given program 
encompass a wide array of factors. These clearly include 
but extend well beyond pedagogical issues such as 
curriculum content and currency. For example, 
resources such as buildings, computing facilities, 
availability of internships, student/teacher ratios, and 
faculty expertise, as well as services such as academic 
advising, career planning, and job placement come into 
play. Behavioral and attitudinal factors may also be 
included. In particular, changes in student attitudes 
(tolerance, respect for others, valuing diversity, etc.) 
have been identified as important outcomes in the 
education literature (Travis, 1996; Haksever and 
Muragishi, 1 998). 

The purpose of this study is to identify the strengths and 
weaknesses of an IS degree program as viewed by 
graduating seniors and by alumni. The results will be 
used in a larger study aimed at the development of a 
performance assessment model that can be used for 
continuous evaluation of IS programs. 

METHODOLOGY 

In an effort to address some of the issues on program 
effectiveness as they relate to the BBA with a major in 
Information Systems, current and former students were 
surveyed on the strengths and weaknesses of the 
information systems program /curriculum at a public, 
regional, comprehensive university in the southeastern 
United States. The groups surveyed during the first 
iteration of the project were those enrolled in senior- 
level IS courses (CISM 4xxx) during Spring 2000 and 
alumni who graduated in 1998 and 1995. One goal of 
the exercise is to determine if these groups identify 
similar or dissimilar strengths and weaknesses. The 
results of the survey will become one component of an 
ongoing program assessment. 
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Data collection for this study was facilitated by the 
availability of tables from our university’s data 
warehouse. The university’s data warehouse includes an 
alumni table containing information on alumni such as 
name, degree, major, graduation date, home address, 
home phone, home e-mail, job-code, business phone, 
address, etc. One ne gative of the table is that it does not 
have complete information on all those in the table. 
However, the table did provide us with a setof data from 
which alumni, by year, could be identified and 
contacted. Given the relatively small number of alumni 
in the two years of interest (69 graduates in 1998, 41 
graduates in 1995, for a total of 1 10 in all), a decision 
was made to mail a survey to each and every one of 
these alumni, rather than selecting arandom or stratified 
sample. Follow-up attempts were later made to contact 
these alumni via phone and/or e-mail (when such 
information was available) to encourage participation. 

The survey instrument for this study was developed 
during the Spring 2000 semester and used to survey 
students enrolled thatterm in senior-level (CISM 4xxx) 
courses. (See Appendix A.) The instmment was 
subsequently used for the alumni survey during summer 
2000. It should be noted that a system-wide change 
from academic quarters to semesters occurred in the Fall 
of 1998. As a result of this transition, all courses at the 
university were renamed and renumbered. Thus, courses 
are identified on the survey instrument by both their 
quarter-system and semester-system equivalents. 
Concurrently with the quarter-to-semester transition, a 
few courses that had previously been required in the IS 
curriculum (such as Applications Development with 
COBOL) became electives, and some that had 
previously been electives(such as Information Resource 
Management) became required. Several new courses 
(such as Project Management) were added, and others 
(such as the introductory programming course) were 
significantly revised andupdated in terms of the content 
and/or tools being used. (For example, the language 
used in the introductory programming course shifted 
from Pascal to Java). 

Because this is an exploratory study, the survey 
instrument was purposefully designed to contain only 
open-ended questions. Although a few of items that 
could be considered as strengths or weaknesses were 
given as examples, only blank lines were given for the 
respondents to fill in. This allowed for the widest 
possible range of responses, from which patterns would 
hopefully em erge. 



RESULTS 

Response rates from both the student and alumni groups 
were very low. Of the 8 1 students asked to particip ate in 
the Spring of 2000, only 13 (or 16%) completed the 
survey. Results for alumni were, unfortunately, even 
lower. Five of the 110 surveys sent to alumni were 
returned as “undeliverable”. Of the 105 remaining, only 
8 (or 7.6%) responded. The poor response rate among 
alumni is partially attributable to the poor quality of the 
university’s alumni database. This became apparent as 
follow-up attempts to encourage participation were 
made. Phone numbers an d email addresses were seldom 
available, and when available, were often wrong. While 
we believe the “personal touch” would have improved 
the response rate significantly, we also realize that IS 
graduates tend to change jobs (and locations) frequently, 
seldom notifying the university of such moves. As a 
result, we believe thatthe quality ofthe alumni database 
will continue to be problematic, and consequently, we 
will need to find (or develop) better sources of alumni 
data for future studies. On the other hand, we believe 
that the response rate among seniors can be improved 
dramatically by asking these students to complete the 
survey at the same time that they complete evaluations 
of teaching effectiveness. Student evaluations of 
teaching effectivenessare given in all classes every Fall 
and Spring semester. We plan to use this approach in 
our next data gathering cycle. 

The data collected for this study were analyzed with 
basic, categorical data analysis techniques. Tables 1 
through 5 summarize the results for each of the five 
questions. 

DISCUSSION 

Strengths 

An examination of Table 1 reveals that alumni and 
seniors view the strengths ofthe IS program similarly in 
most respects, with a few notable exceptions. Both 
groups view faculty as the dominant strength, both in 
terms of the quality of the faculty and their 
approachability, availability and concern fo r the studen t. 
Both groups also view courses as a strength of the 
program, but they differ in terms of the “focus” seen as 
important within those courses. Alumni viewed the 
foundation in business and the emphasis on long-term, 
underlying principles as strong points, while seniors 
indicatedstrengths in terms ofan emphasis onteamwork 
and hands-on projects. While it cannot be determined 
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from the data gathered, it is possible that the emphasis 
on teamwork and projects indicated by the seniors 
represent some of the “long-term principles’’ indicated 
by alumni. 

Both groups saw lab hours as a strength. Interestingly, 
however, the two groups differed substantially on their 
views of the lab equipment. Alumni viewed the labs as 
a strength to a much greater extent than did the students. 
This may suggest that alumni develop a greater 
appreciation for the quality of the labs once they have 
worked in the field and had more exposure to other 
computer facilities and operating environments. 

Internship opportunities were noted by both groups as a 
program strength, however, only seniors indicated 
strengths with respect to employment opportunities and 
job placement. Seniors mentioned guest speakers as a 
strength much more frequently than did alum ni. Seniors 
also noted support staff, classroom technologies, class 
size, and the overall breadth and direction of the program 
as strengths; these were not noted by alumni. 

Weaknesses 

While the two groups viewed strengths similarly, they 
viewed weaknesses very differently. (See Table 2). 
None of the alumni who responded noted any 
weaknesses in terms of the faculty. A few weaknesses 
relating to faculty were, however, noted by seniors. 
These included the English-speaking skills of some 
faculty, failure tomatch courses to faculty strengths, and 
examination and evaluation policies utilized by some 
faculty. 

Interestingly, while “courses” were noted as a strength 
by both groups, they were also noted (in fairly equal 
numbers) as a weakness by both groups. Students and 
alumni alike indicateda need for more hands-on projects 
and continuing effort to keep course topics up-to-date. 
Seniors repeatedly mentioned the need for more sections 
of courses, particularly more sections of IS electives; 
alumni, however, did not mention this at all. Seniors 
also indicated a desire for fewer business courses and 
more IS courses (which is somewhat at odds with the 
“focus on business” strength noted by alumni), as well as 
a desire to be allowed to choose a “focus area” within IS. 



While respondents in each group viewed internship 
opportunities as a strength, several alumni indicated this 
as a weakness, suggesting that more internship 
opportunities need to be made available. This may be 
due to the fact that alumni are more likely to recognize 
the value of an internship experience. Both groups 
indicated a need to provide more encouragement for 
students to complete internships. Along similar lines, 
alumni suggested a need to develop more networking 
and/or job shadowing opportunities with IS 
professionals, as well as a need to provide more guest 
speakers; these issues were not mentioned by seniors as 
weaknesses, although seniors did express a desire for 
more help in finding internships and jobs. 

A final weakness mentioned by seniors but not by 
alumni had to do with staffing. Specifically, seniors 
mentioned a need for more tutors, more faculty, and 
more lab assistants. 

TABLE 1 
QUESTION 1: 

STRENGTHS OF THE IS PROGRAM 

Number of Times 
Mentioned 



By By 
Alumni Seniors 

Faculty 

General (quality, knowledge, etc.) 7 12 

Approachability, availability, concern, 
etc 2 4 

Courses 

General 3 6 

Focus on business, long-term principles 2 

Focus on teams 3 



Focus on hands-on projects 
Class size 
Curriculum 



Overall program breadth & direction 3 

Labs 

Equipment 5 3 

Lab hours 3 5 

Internships 

General 2 

Internship opportunities 2 2 

Employment opportunities/job placement 2 

Guest Speakers 1 5 

Support staff 1 

Technology in classrooms 1 
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TABLE 2 
QUESTION 2: 

WEAKNESSES OF THE IS PROGRAM 

Number of Times 
Mentioned 



By By 

Alumni Seniors 

Faculty 

General 1 

English-speaking skills 2 

Not matching courses to faculty 1 

strengths 

Testing/grading policies 2 

Courses 

General 1 

More hands-on/teach in labs/projects 3 5 

Need to update/more hot topics 2 2 

More time for projects 1 

More sections of electives needed 7 

Curriculum 

Include less business courses, more IS 1 

Not allowed to have focus area within 1 

IS 

Labs 

Equipment not working properly 6 

Equipment in networking lab 1 2 

Availability to IS student 1 5 

Internships 

Opportunities 3 

More encouragement/requirement to do 1 2 

Employment opportunities/job placement 
Networking/job shadowing 

opportunities 3 

Help in finding internships &jobs 1 

Guest Speakers 4 

Staff 

Support staff 1 

Tutors needed 1 

More faculty needed 1 

Not enough lab assistants 1 



Most and Least Valuable Courses 



Systems Analysis and Design tying for second place. 
This suggests th at both groups recognize the importance 
of the underlying concepts and principles that comprise 
the fundamental core of the IS curriculum. 

There is a much wider disparity ofopinions with respect 
to the least valuable course, particularly among seniors. 
This is quite possibly due to the fact that many seniors 
simply do not know yet where their careers will lead 
them and have little or no experience upon which to base 
their answers. The course most frequently mentioned as 
“least valuable” by seniors was Data Management. This 
perspective is not shared by alumni, however, none of 
whom mentioned this course as “least valuable” and 
many of whom mentioned it as “most valuable”. The 
overwhelmingly “least valuable” course to alumni was 
COBOL. COBOL was m entioned as “least valuable” in 
more of the alumni responses than the senior responses. 
This supports the trend in industry to move away from 
COBOL for new development projects, and is no doubt 
influenced in this study by the factthat COBOL was (but 
is longer) a course required in the IS core. Another 
interesting difference in the least valuable course centers 
on the other programming course (Applications 
Development with Java). Relatively speaking, fewer 
seniors than alumni view this course as “leastvaluable”. 
This difference of opinion may come about as a result of 
the “normal” career path taken by IS graduates, many of 

TABLE 3 
QUESTION 3: 

WHICH COURSE (CONTENT) CONTRIBUTED 
THE MOST TO PREPARE YOU FOR YOUR 
FUTURE IN THE IS FIELD? 

Number of Times 
Mentioned 
By By 
Alumni Seniors 



Tables 3 and 4 summarize the viewpo ints of seniors and 
alumni with respect to the most and least valuable 
courses in the IS curriculum . The two groups were very 
similar in terms of the course that contributed the most 
to their preparation, but differed with respect to the 
course seen as “least valuable”. Courses in the “heart” 
of the IS core (Applications Development, Systems 
Analysis and Design, Data Communications, and Data 
Management) were the most valuable to both groups. 
Interestingly, seniors indicated Systems Analysis and 
Design as the “top” course, followed by Data 
Communications, then Data Management. This order 
“flip-flopped” with alumni, who gave the top honor to 
Data Management, with Data Communications and 



CISM3130 

CISM3134 
CISM3135 
CISM4134 
CISM4135 
CISM4136 
CISM 4234 

CISM 4235 

CISM 4236 
CISM 4237 
CISM 4238 



(Applications Development 
with Java) 

(Data Communications) 
(Systems Analysis Design) 
(Data Management) 

(Project Management) 

(Global Info. Resource Mgmt.) 
(Applications Development 
with Objects) 

(Applications Development 
with COBOL) 

(AS 400 and Its Applications) 
(Decision Support Systems) 
(Network Administration) 



1 2 

2 5 

2 7 

3 3 
1 



1 

1 



1 1 



Note: Some participants mentioned more than one course in their 
response to this question. 
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TABLE 4 
QUESTION 4: 

WHICH COURSE (CONTENT) DO YOU 
BELIEVE WILL BE THE LEAST VALUABLE 
TO YOU IN YOUR CAREER? 



None 
Not sure 
CISM3130 

CISM3134 
CISM3135 
CISM4134 
CISM4135 
CISM4136 
CISM 4234 

CISM 4235 

CISM 4236 
CISM 4237 
CISM 4238 
Note:Some 



Number of Times 
Mentioned 
By By 
Alumni Seniors 



2 

1 

(Applications Development with 
Java) 2 2 

(Data Communications) 1 

(Systems Analysis and Design) 1 3 

(Data Management) 5 

(Project Management) 1 

(Global Info. Resource Mgmt) 2 

(Applications Development with 
Objects) 

(Applications Development with 

COBOL) 3 1 

(AS 400 and Its Applications) 

(Decision Support Systems) 1 

(Network Administration) 1 1 



participants mentioned more than one course in their 
response to this question 



whom begin as programmers but quickly move into 
other specialty areas within IS. Alumni may view 
programming courses in general as “less valuable” s a 
result of making this transition in their own careers. 



Course in Which You Learned the Most 



The Data Management course is viewed as the winner in this 
category by both alumni and seniors. Othercourses frequently 
mentioned by both groups include those in introductory 
programming, datacomm unications, and systems analysis and 
design. 



LIMITATIONS 

The results of this study are obviously limited by the low 
response rate. When viewed as a pilot study, however, 
the results are nonetheless valuable. The results will be 
incorporated into the next iteration of this study to (1) 
improve the content of the instrument and (2) increase 
the response rate. By repeatingthis study each semester 
and building an on-going data set, we hope to develop a 
performance assessment model that can be used in other 
similar IS programs. As data are eventually collected 
across multiple universities, a better understanding of 
performance assessment measures should emerge. 



TABLE 5 
QUESTION 5: 

COURSE IN WHICH YOU FEEL 
YOU LEARNED THE MOST 

Number of Times 
Mentioned 



By By 

Alumni Seniors 

CIS 381 (Introduction to Programming) 1 1 

CISM 3 1 30 (Applications Development 

with Java) 2 

CISM 3134 (Data Communications) 2 2 

CISM 3135 (Systems Analysis and Design) 1 1 

CISM 4 134 (Data Management) 4 5 

CISM 4135 (Project Management) 1 

CISM 4136 (Global Info. Resource Mgmt) 1 

CISM 4234 (Applications Development 

with Objects) 1 

CISM 4235 (Applications Development 

with COBOL) 1 

CISM 4236 (AS 400 and Its Applications) 

CISM 4237 (Decision Support Systems) 1 

CISM 4238 (Network Administration) 1 



Note: Some participants mentioned more than one course in their 

response to this question 



SUMMARY 

This paper has reported the results of an initial 
performance assessment study undertaken to identify the 
strengths and weaknesses of an IS program as viewed by 
alumni and graduating seniors. There were more 
similarities between groups than differences, particularly 
with respect to strengths of the program. Major 
strengths identified by both groups included faculty, 
courses, and lab facilities. Weaknesses identified most 
frequently by seniors included the need for more sections 
of electives and better/more lab facilities. Primary 
weaknesses noted by alumni included the need for more 
interaction with IS pro fessionals and the need to keep the 
curriculum up-to-date. This paper represents an initial 
step in an on-going study. The results will be used to 
refine and improve the methodology, and ultimately, to 
devise a useful model for performance assessment in 
university-level IS programs. 
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Information Systems 
Senior Courses Survey 
May 2000 



Dear CISM Students: 

As I hope that you know, the College of Business Administration at University cares about its students and 

tries very hard to offer programs that are current and meeting the needs of its students as they prepare to m ove into 
careers and graduate school. In orderto accomplish that goal, we need help from our current students and our alumni. 
This survey is being sent to each student enrolled in CISM 4xxx courses during Spring 2000 and a random sample 
of alumni who were graduated in 1998 and 1995. We plan to use the results as one component of an ongoing 
assessment of our program. 

Please be assured that your responses will be handled confidentially by me. If you would prefer, you may print this 

survey, complete it by hand, and deliver it to me personally, to Ms. or Mrs. in the 

Department Office, or via campus mail at P. O. Box . Or, you may reply in this attachment and either attach it 

to an e-mail to me at or drop it into the CISMSurvey Folder in my dropbox on the k drive using any 

anonymous filename you want. 

However you reply, please do so by next Friday, May 1 2, 2000. 

Thank you for your help and for being a part of our program. 

Sincerely, 



Chair, Department of 



Please go to the next page. 
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Below you will find a list of the courses which have been offered as part of the Information Systems 
curriculum during the past three years. 



Semester Courses 

CISM 3130 Applications Development 
CISM 3134 Data Communications 
CISM 3135 Systems Analysis and Design 
CISM 4134 Data Management 
CISM 4135 Project Management and Development 
CISM 4136 Global Information Resource 
Management 



Quarter Course Equivalents 



CIS 381 Introduction to Computer Programming 
CIS 484 Data Communications 
CIS 384 Systems Analysis and Design 
CIS 488 Applied Database Systems 



CIS 486 Information Resource Management 



CISM 4234 Application Development with Objects 

CISM 4235 Application Development with COBOL CIS 382 COBOL Programming 
CISM 4236 AS/400 and Its Application 
CISM 4237 Decision Support Systems 
CISM 4238 Network Administration 



Please take a few moments to recall the courses from the list above which you completed during your time 
at GSU. Consider the content and the value of that content in preparing you for your future career. 

Which course (content) contributed most to prepare you for your future in the IS field? 



Which course (content) do you believe will be least valuable to you in your career? 



In which course do you feel you learned the most? 



Please go to the next (and last) page. 
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List three to five strengths of the Information Systems Program: 

(For example, internship opportunities, outside speakers, faculty, courses, equipment, laboratory 
hours, support staff, .... you tell us!) 



List three to five weaknesses (Opportunities forimprovement) oftheinformation Systems Program: 



Finally (and thanks for getting this far), please share anything else you feel that we should know 
about the Inform ation Systems Pro gram an d, if you would like to add toyour comments in the future, 
we always welcome your input. 

Thanks. 



Proceedings of the 15'*’ Annual Conference of the International Academy for Information Managemoit 





TOWARDS A KNOWLEDGE 
MANAGEMENT MODEL FOR THE 
INFORMATION MANAGEMENT CURRICULA 
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Manchester Metropolitan University 



Ray Hackney 

Manchester Metropolitan University 



ABSTRACT 

The growth of interest in all things 'knowledge management* (KM) is exponential. Developments of products and 
ideas, fuelled by a newly designated knowledge community, is happening at such speed that few seem to question the 
trends or the knowledge management systems findingfavour in organizational life. A consideration ofwhat is being 
presented as KM notes its remarkable similarity to the traditional features of 'information management ' (IM) re- 
dressed in appearance but perhaps not the ideal starting point to get us to the desired destination. This paper 
identified these trends and develops prescriptions for the IM curricula. Most importantly it offers a critique of some 
of the issues that appear frequently absent in the teaching and learning of KM, 



INTRODUCTION 

Conceptually, KM itself may not be so very different 
from IM for it certainly appears to have some roots in 
organisational learning and in the concept of learning 
organizations. The advent of the Learning Organization 
(Argyris 1982, Senge 1990, De Geus 1997) or the 
Learning Company (Pedlar et al 1991) appears more 
than merely another management theory fad that 
promises unattainable benefit. In a review of these 
literatures one of the many disciplinary perspectives is 
management science with its ontology of information 
creation, capture, storage and dissemination. A role for 
the IM discipline in this endeavour is self-evident as is 
the development of supporting organisational 
infrastructure based upon information technology (IT) 
platforms and solutions. This disciplinary interest can 
also be tracked through Information Systems (IS) 
strategy literature in which surveys have consistently 
identified organisational learning as a key and strategic 



goal for the IT function. This goal has a recurring 
prominence over the last two decades. More recently a 
significant development within these literatures is 
focused around the concept of KM (Ranchhod and 
Hackney, 2000). Whilst KM is nota new theme, with its 
roots traced to the ancient Greeks, the contemporary 
opportunities afforded through IT developments would 
seem to open many previously incomprehensible 
pedagogic opportunities. Consequently, growth in 
interest in KM theo ry and practice is illustrated through 
the increasing prominence of the theme in the IM 
curricula. The increase in conferences, products and 
services are also being offered in support of the KM 
endeavour illustrates the probability of this theme being 
substantial and gaining much more significance than 
even its current highly visible status. These increases 
seem to coincide with a reported decline in the 
publication output around organisational learning. 
Indeed, during the 90’s that decline in journal 
publications corresponds with an incline in those 
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concerning KM, One interpretation of this mightbe that 
KM has become the current organisational learning 
theme. One has not replaced the other; rather they are 
part of the same genre. The currency of the KM theme 
though particularly strong and still strengthening might 
at some stage nonetheless be replaced by a yet more 
contemporary theme, though both still part of the 
organisationallearningparadigm. Presently what we can 
observe is that the teaching and learning of KM is of 
such significance and interest that deeper pedagogic 
research is both warranted and worthwhile. 

THE ROLE OF STUDENT INTRANETS 

Intranet media designs offer support for traditional 
learning and, more critically, they augment the 
development of new ways of learning. This is to suggest 
that an Intranet provides the opportunity for different 
means of delivering information, different means of 
human-computer interaction and different means of 
creating appropriate environments for on-line learning. 
Clearly, there are a number of complex issues 
surrounding the achievement of these objectives which 
extend beyond a simplistic ‘meet the needs of students’ 
design. The available tools, including multimedia 
authoring software, Java integrated environments, 
common gateway interfaces, artificial intelligence, 
virtual reality modelling and language creation 
interaction techniques, all offer possible solutions. 
However, what is needed in this respect is a 
determination of the cognitive styles and preferences of 
the students in their interaction with an Intranet, 



Consequently, it may be possible to superimpose more 
general design principles onto the cognitive profiles of 
particular kinds of students. This should enable the better 
use of an on-line community for the realisation of an 
advanced learning experience (Hackney and Pagano, 
2000 ), 

Experience from KM could be viewed as a critical 
success factor in this respect for students who must 
continually learn if they are to survive in contemporary 
business (DeGues, 1997), Within these strategies it is 
clear that Intranets have a leading role for they pervade 
many areas of organizations and beyond to other 
elements ofthe supply chain (Millerand Dunn 1 998), In 
particular. Intranets are increasinglyexpectedto provide 
the knowledge dissemination infrastructure inter and 
intra organizationally, so as to support learning 
activities. Any comprehensive survey of the learning 
organization literature and practice quickly reveals the 
significance of KM for it is through the capturing of 
information and sharing of knowledge that organizations 
can be seen to learn (IPD, 1999), The inference is clear 
that for a successful teaching and learning strategy KM 
necessitates close attention to the issue of creating 
organizational know ledge and of its shared management, 
KM can be said to be the policies andprocesses through 
which organizations seek to create, store and dissem inate 
organizationalknowledge,and Intranets are fundamental 
to this endeavour (Dick and Burns, 1999), In this way 
the virtual environment can be understood to be a 
comprehensive knowledge system, as shown in Table 1 , 



TABLE 1 

KM CURRICULA KNOWLEDGE SYSTEM (ADAPTED FROM ENDLAR, 2000) 





Teams 


Technology 


Learning 


Trust 


The building of positive team 
processes 


Open and accessible 
information 


Implicit knowledge made explicit 
through sharing of information 


Conflict 


Building relationships and mixing 
team talen ts 


Open and instant 

communications for working 
with conflict 


For deep levels of discussion and 
dialogue 


Dialogue 


A “core competency” for developing 
effectiveness and facilitating 
interaction 


Enables a “higher order” of 
communication 


Open and powerful communi-cation 
for moving beyond single 
understandings 


Meetings 


Helps motivate individuals while 
building relationships and “shared 
vision” 


Asynchronous and 

synchronous meetings 


Synchronised action witho ut specific 
action plan 


Electronic 

Practice 

Fields 


Practice is performed in similar to 
work situations 


Establishes real work 
environments for practice 


Learning occurs through team 
processes without the fear of 
consequen ces 


Technology 

Teams 


Greater performance levels attained 
with teams using technology 


Technology enables teaming 
processes of learning 


Captured team knowledge results in 
learning for new team members and 
the organisation 
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Table 1 
(continued) 





Teams 


Technology 


Learning 


Virtual 

Teaming 


Real life skills and new team skills 
for enhanced learning 


Teaming without face-to- 
face contact 


Continuous learning possible 
without waiting for next meeting 


Networks 


Develop team members 


Enables instant access to 
information 


Implicit knowledge made explicit 
through sharing of information 
between team 


Groupware 


Necessity forbuilding relation-ships 
and bonding team members 


Enables instant communi- 
cation for dealing with 
conflict 


Needed for reaching deep levels of 
discussion and dialogue 


Collaboration 


Team vision and focus for the team 


User driven technological 
development enhanced 


Learning opportunities from failure 


Electronic 

Practice 

Fields 


Practice is performed in similar to 
work situations 


Establishes real work 
environments for practice 


Learning occurs through team 
processes without the fear of 
consequences 


Technology 
T earns 


Greater performance levels attained 
with teams using technology 


Technology enables teaming 
processes of learning 


Captured team knowledge results in 
learning for new team members and 
the organisation 


Virtual 

Teaming 


Real life skills and new team skills 
for enhanced learning 


T earning without face-to-face 
contact 


Continuous learning possible 
without waiting for next meeting 


Networks 


Develop team members 


Enables instant access to 
information 


Implicit knowledge made explicit 
through sharing of information 
between team 


Groupware 


Necessity for building relation-ships 
and bonding team members 


Enables instant com muni- 
cation for dealing with 
conflict 


Needed for reaching deep levels of 
discussion and dialogue 


Collaboration 


Team vision and focus for the team 


User driven technological 
development enhanced 


Learning opportunities from failure 


Team 

Learning 


To align and develop teams to create 
results by challenging assumptions 


Immediate and continuous 
dialogue and sharing work 
whilst apart 


Collective intelligence that is greater 
than the some of individual 
intelligence’s of the team members 


Knowledge 


Information transformed in 
knowledge 


Tools for collaboration for 
capturing and storing 
knowledge 


Greater learning leads to more 
knowledge in organisation 


Implicit/ 

Explicit 

knowledge 


To build learning teams implicit 
knowledge must become explicit 
knowledge 


Sharing of knowledge 
increases transfer of im plicit 
knowledge to explicit 
knowledge 


Implicit knowledge made explicit 
and shared 



Contemporary applications of KM systems therefore 
utilize Intranet as their principal tool. As oneseniorKM 
officer of a global top-six computer company recently 
asserted, “Our company wide Intranet is the window on 
the organizations know ledge”. It is through this window 
that our students may learn about their increasingly 
virtual business world. 



INFORMATION MANAGEMENT VS 
KNOWLEDGE MANAGEMENT TEACHING 

There also appears considerable confusion, both in the 
literature and in organisational practice, between KM 
and IM. The two are often considered to be the same 
thing though it is clear that on more precise scrutiny they 
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are not. If knowledge were the same as information then 
we would not need a different name in language to 
define it. Information and the data from which it is 
derived can be captured, stored and disseminated with 
relative ease and through commonly accepted practice. 
In effect, information is organisational history, codified 
and managed through information technology 
infrastructures amongst others. Conversely knowledge 
focuses not on history but on future. Knowledge 
creation is an outcome of human cognition that is unique 
to the individual and therefore inevitably influenced by 
phenomenology. Knowledge provides the capacity for 
individuals to take action, it is the ‘know how’ to 
information’s ‘know what’. Knowledge therefore cannot 
be captured in the same way as information nor managed 
accordingly. Indeed, knowledge cannot be managed at 
all, which means the term KM is regrettably misleading. 
Like many accepted and established labels in language 
KM is commercially expedient rather than accurately 
indicative of the endeavour itself. Even within the IM 
literature, knowledge is frequently positioned to follow 
information without the recognition that the two are in 
different domains, one retrospective, and the other 
futuristic. Understanding organisational knowledge 
might mean to understand how to bridge the gap 
between these two domains. This ‘continuum’ between 
organizational information and organizational 
knowledge appears previously to have received little 
discernable attention. 

It is useful, there fore, to identify two critical curriculum 
issues for students of KM; 

1. KM issues remain largely ambiguous or 
misunderstood with different organisational 
responses in evidence. Practice is as varied as 
organizations definitions. One of the few common 
responses, particularly amongst large organisations, 
appears substantialcommitment in resource termsto 
the issue. KM products are flourishing; KM people 
are being appointed, sometimes at very senior levels 
(CKO’s). Spending on KM appears to be taking an 
increasing proportion of IS organisational budgets. 
From all of these investments in time, people and 
systems, expected and anticipated paybacks will be 
substantial. Vendors offering KM solutions are 
likely to gain considerably in the next decade, 
perhaps more profitably than their clients in whose 
organizations their solutions will be deployed. IS 
strategy responses to KM appear largely to be 
around creating knowledge repositories (databases) 



and groupware applications, typically Lotus Notes 
and Intranets. 

2. Discussion around knowledge capture, codification 
and dissemination too often simply mirrors the 
process of information management. The issue that 
in fact knowledge might not be capturable at all 
seems to be largely overlooked or ignored by many 
organisations. Further, that knowledge is not 
manageable like otherorganizational assets appears 
outside the realisation mindset of many. Distinct 
from the IM field, the human resource management 
(HRM) response to KM appears largely to be 
concerned with employee retention as a means of 
retaining knowledge. In an area of declining 
employment longevity and in which careers are now 
largely a series of consecutive steps between 
organisations, rather than within the same 
organisation, such strategy has limited and perhaps 
unrealistic value. Neither the IM nor the HRM 
functions seem to have yet developed an adequate 
strategic response to KM. The IPD have recently 
expressed this in terms of wishing KM to be 
perceived as an HRM issue for organisations whilst 
recognising the IT function islargely responsible for 
developing it and thus far maintaining the high 
ground. It appears time for new thinking around 
these issues, for neither IM nor HRM are likely to 
adequately respond to KM from their relative 
positions of isolation. 

CONTEMPORARY CURRICULUM ISSUES 

Contemp orary KM feels intuitively almost a new issue; 
such is the interest and enthusiasm surrounding its 
apparent potential. KM promises opportunity not 
previously so well developed in oiganisational life. The 
facilitator of this opportunity is undoubtedly IT and 
within that, developments in communications 
technologies and software applications. However, the 
value added is predominantly to its people who learn, 
and use that learn ing to operate m ore effective ly both as 
individuals and in their capacity to assist others within 
the organization. Consequently, within these issues, 
most notably organisational culture seems to make the 
significantdifferencebetween success andfailure inKM 
endeavour. We can work with knowledge but not 
manage it, though we can to some extent manage the 
environment, manage the people and manage other 
organisational factors. In doing so we stay consistent 
with an aspiration to build a quality learning 
environment. 
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From teaching practice, four generic strategies can be 
discerned from reported KM endeavours (adapted from 
Prusak, 2000). 

1 . Knowledge replication i.e. banks, fast food chains, 
retailers e.g. Toys ‘r’ Us, MacDonald’s, INTEL etc, 
all provide examples of conducting operations in 
precisely the same way wherever they are located 
world wide. 

2. Knowledge diffusion/leveragability is about 
“knowing what we know, andusing it”. This seems 

to be the key strategy in most KM endeavours. 

/ 

3. Knowledge innovation, which is concerned with 
knowing what comes nexte.g. new products, new 
services, new ideas that might keep an 
organisation competitive. 

4. Knowledge commercialisation, which is concerned 
with what does the organisation know that it can 
sell, i.e.. Consultancy, products and services, e.g. 
British Gas pic ha ve developed KM systems initially 
for their own organisational needs but which they 
now seek to promote externally. 

Ofthese generic strategies, knowledge diffusion appears 
the most prolific. However, organisations probably 
pursue more than one, perhaps sometimes all of these 
strategies simultaneously. By way of illustration a 
pharmaceutical company might be interested in 
strategies 1, 2 and 3 whilst a consulting group might 
focus on strategies 2 and 4. By segmenting the KM 
issue against differing strategic intention our curricular 
might develop a clearer perspective on the real 
contribution of KM. Strategically, we could be 
optimistic and feel that KM might afford ourstudents the 
opportunity of acquiring competence towards 
competitive advantage. Realistically, we can predict 
with greater certainty that an inadequate teaching of KM 
will almost certainly leave individuals with a 
competitive disadvantage. 

CONCLUSION 

KM presents a different set oforganisational challenges 
and appears to push the IM paradigm boundary 
sufficiently to require a different epistemological 
perspective. Participation instudent group activities and 
events supports the articulation and use of tacit 
knowledge can never be captured or codified even if 
organisations had unlimited resources in which to make 



the attempt. Communities of people, networks and 
groups can share practice, knowledge and experience, 
which then become embedded in organisational routines 
and ultimately enculture the organ isation. KM therefore 
may very well present new forms of organisational 
opportunity in the future. Taking the time to debate the 
more difficult aspects of KM might slow down the 
adoption of a KM fagade. Whilst this might have short 
term unpopularity amongst vendors selling KM 
‘solutions’ the long term benefits associated with 
breaking new ground, delivering the promise and 
potential of leveragin g organizational knowledge, might 
be significantly rewarding for our students. 
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TEACHING INFORMATION SYSTEMS 
AND INTERNATIONAL STUDENTS: 

A QUALITATIVE EXAMINATION OF THE 
CULTURAL DIFFERENCES IN COMPUTER 
SUPPORTED GROUP WORK 



Nasrin Rahmati 
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ABSTRACT 



This study explored the Impact of group technologies on student group work. The study focused on the differences 
on the impact of Group Support Technologies between students from two different national cultures. Groups of 
students from Australia and Malaysia participated in a series of GSS supported experiments using an unstructured 
task. There were four to five members in each group. Qualitative analysis of their comments showed that there were 
differences between groups from the two national cultures in the values they referred to in decision making process 
and further differences between the groups when they moved from ordinary group work to computer supported 
groupwork. In addition these differences were not alike for the groups from the two nations. The use of GSS 
technologies caused comments to be closer to the stated individual values by the participants. 



INTRODUCTION 

Rapid developments in information technology (IT) have 
generated potential changes to teaching and learning. In 
particular, technology-mediated learning which is the 
type of learning which involves implementation of 
information, computing, and communications 
technology applications (Alavi, Wheeler, & Valacich, 
1995) in more than one location. 

Technology-mediated learning of subjects is becoming 
an important option within IS education because it 
facilitates the sharing of costs, information, and expertise 
among multiple sites. Another advantage of using 
information technology in education is that students are 
introduced to and take advantage of the very 
technologies that businesses are using to gain 
competitive advantage (Leidner & Jarvenpaa, 1993). 



Most of the schools are under increased pressure to 
graduate students with experience with these emerging 
technologies (Alavi et al., 1995, Webster & Hackley, 
1997). 

Most of the information systems subjects in different 
schools provide access to different types of technologies 
to facilitate on-line groupwork. These might be as 
simple as a shared whiteboard, chat room or as 
sophisticated as the group decision support systems 
technologies. Some of these technologies can be set up 
in such a way to provide the students with complete 
anonymity. While for submitting a comment, some of 
the work group technologies require turn taking the 
others provide simultaneous entry of comments. In some 
cases lecturer/moderator can arrange a time for all of the 
students to be present on-line for the discussion while in 
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other cases the discussion can take place not only at 
different places but also at different times. 

Little research has addressed the impact of these 
technologies on students group work. As Leidner and 
Jarvenpaa (1993) and Alavi and colleagues (1995) 
outlined in their reviews of the learning literature, 
learning is best accomplished through the active 
involvement of students. The degree and the quality of 
student involvement in a decision making situation, 
according to the past cultural studies, depend on their 
cultural values (Yates & Lee, 1996). 

The present study is an attempt to examine the interplay 
among cultural values, technology and the group work 
processes. The present study reviews the past cultural 
studies (eg, Hofstede, 1980; Schwartz & Bilsky, 1987; 
Schwartz, 1992), reports on a cultural survey and a series 
of quasi laboratory experiments. The attempt is to find 
a model for student groupwork based on their cultural 
values. 

PAST CULTURAL STUDIES 

Triandis (1994) believes that culture is to society what 
memory is to individuals. He believes that culture 
provides traditions that tell people what has worked in 
the past and makes it easy for humans to pick behaviours 
that may work again in the present Kluckhohn (1954) 
and later Triandis (1994) suggested that culture 
influences the way humans select, interpret, process, and 
use information. It is suggested by many cross-cultural 
researchers that culture includes systems of values (e.g., 
Hofstede, 1980; Mead, 1994). Cultural values are 
passed on by other members of the culture group, and 
become second nature and massively influence the 
individual’s behaviour (Berry et al., 1992). They 
determine how the individual interprets the context of 
events; what is selected as important in a context, what 
needs to be explained and what can be taken as routine, 
or edited out of consciousness as insignificant. 

Hofstede (1980) defines culture as ‘the collective 
programming of the mind which distinguishes the 
members of one category from the other’. As Hodgetts 
et al. (1994) state '‘Culture can affect technology transfer, 
managerial attitudes, managerial ideology.... and how 
people think and behave.’ 

Some researchers have tried to provide a complete 
picture of national culture by dividing the culture into its 
subgroups and call them cultural dimensions. As 



Triandis (1994) states this is a valid way of unpacking 
culture to make it measurable. Hofstede suggested three 
cultural dimensions Power Distance, Uncertainty 
Avoidance and Individualism. He later extracted 
Masculinity/Femininity out of Individualism. Although 
his work has been recognised as a major cross-cultural 
work, the findings belong to two decades ago and there 
seems to be no simple way to tie his findings to any 
organisational activity. 

Hofstede 

Through what Hofstede (1980) called an ecological 
factor analysis of 32 mean scores for each of the 40 
countries he identified three cultural dimensions, one of 
which was further split into two components, producing 
four dimensions altogether. Hofstede concluded that 
together these four dimensions explain half of the 
differences in mean value scores among his sample of 40 
national cultures. 

The labels Hofstede (1980) selected for the four 
dimensions together with their interpretations are as 
follows; 

• Power Distance: Hofstede (1980) defines ‘Power 
Distance’ as ‘... the extent to which the less 
powerful members of institutions and organisations 
accept that power is distributed unequally’. Power 
distance refers to the extent to which unequal 
distribution of power in institutions and 
organisations is accepted by members of a society. 

• Uncertainty Avoidance: The extent to which 

members of a society feel threatened by uncertainty 
is called ‘Uncertainty Avoidance’ (Hofstede, 1980). 

• Individualism/Collectivism: Hofstede (1980) defines 
individualism as the relationship between the 
individual and the collectivity which prevails in a 
given society. 

• Masculinity versus Femininity: Hofstede (1980) 

states that the predominant socialisation pattern is 
for men to be more assertive and for women to be 
more nurturing. His review of the work goals 
indicated a near consistency on men scoring 
advancement and earnings as more important, and 
women scoring supervision, social aspects of the 
job, working conditions, working hours and ease of 
work as more important (Hofstede, 1980). 
Hofstede’s calculated scores of Masculinity/ 
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Femininity for only a small group of nations are 
available, and for this reason they are not included in 
Table 1 and Diagram 1. 



person are not viewed as distinct from those of the 
group. Conservatism values are primarily concerned 
with security, conformity, and tradition. 



TABLE 1 

HOFSTEDE’S RANKS FOR MALAYSIA AND 
AUSTRALIA IN HIS 52-NATION STUDY 



Country 


Power 

Distance 


Individualism/ 

Collectivism 


Uncertainty 

Avoidance 




Rank 


Rank 


Rank 


Australia 


41 


2 


37 


Malaysia 


1 


36 


46 



Intellectual and Affective autonomy (Two Factors): 
These are the values likely to be important in 
societies that view the person as an autonomous 
entity entitled to pursue his or her individual 
interests and desires. Schwartz (1994) distinguishes 
between two different types of Autonomy values: a 
more intellectual emphasis on self-direction and a 
more affective emphasis on stimulation and 
hedonism. 



Table 1 shows the rank allocated to Australia and 
Malaysia based on Hofstede’s calculated scores in 
comparison with the rest of his sample of national 
cultures. Diagram 1 shows the actual calculated scores 
based on the relative importance (the X axis) for these 
two national cultures assigned to the three cultural 
factors (the Y axis) by participants from Australia and 
Malaysia. 

DIAGRAM 1 

3 CULTURAL FACTORS SUGGESTED BY 
HOFSTEDE (1980) FOR THE TWO 
PARTICIPATING NATIONS 




Schwartz and Bilsky 

Schwartz and his colleague suggested seven cultural 
factors based on a list of 45 values (Schwartz & Bilsky, 
1987; Schwartz, 1992). For each of the samples, the 
mean importance rating of each of these values was 
computed. Instead of any statistical method a 
diagrammatic method called Smallest Space Analysis 
(SSA, suggested by Gauttman, 1968) was used to find 
the cultural factors and their related values. The seven 
factors suggested by Schwartz and Bilsky are as follows: 

• Conservatism: These are values likely to be 
important in societies based on close-knit 
harmonious relations, in which the interests of the 



• Hierarchy: It is suggested by Schwartz (1994) that 
the preference for Hierarchy emerges closer to 
conservatism. The value ‘humble’ falls in this 
region. This location of humble reinforces the 
interpretation of the culture-level hierarchy value 
type as emphasising the legitimacy of hierarchical 
role and resource allocation. 

• Mastery: Mastery values promote active efforts to 
modify one’s surroundings and get ahead of other 
people, whereas Intellectual Autonomy values 
emphasise flexibility of thought and feeling but not 
active social behaviour. ' All the Mastery and 
Autonomous values, and some of the Hierarchy 
values, presume the acceptance of the individual’s 
pursuit of personal interests as legitimate. 

• Egalitarian commitment: These are values that 
express transcendence of selfish interests and are a 
social commitment that can occur in among equals. 
Schwartz believes that these values must be present 
in societies of autonomous individuals to function 
smoothly. 

• Harmony: These values are suggested by Schwartz 
(1994) to emphasise harmony with nature and are 
supposed to be opposite Mastery (it is also stated 
earlier by Kluckhohn and Strodtbeck, 1961). 
Harmony values presume no particular stance 
regarding the autonomy of the person, but they stand 
in opposition to value types that promote actively 
changing the world through self-assertion and 
exploitation of people and resources. 

The seven factors and their relative importance for 

participants of Australia and Malaysia in Schwartz et al. 

Study are represented in Diagram 2. In this diagram the 

‘X’ axis shows the relative importance of each of the 
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seven cultural factors and the ‘Y’ axis identifies each of 
the seven cultural factors. 

DIAGRAM 2 

SCHWARTZ & BILSKY’S 7-FACTOR MODEL 
FOR AUSTRALIA AND MALAYSIA 




A possible relationship between the concepts of 
Hofstede, and Schwartz is presented in table 2. 

TABLE 2 

SUGGESTED EQUIVALENCE 
BETWEEN HOFSTEDE AND SCHWARTZ 



Hofstede 


Schwartz 


Individualism 


Affective individualism 


Collectivism 


Collectivism 


High Power Distance 


Hierarchy 


Low Power Distance 


Social Concern 


Low Uncertainty 
Avoidance 


Intellectual 

Individualism 


Masculinity 


Mastery 


Femininity 


Harmony 



Cross-Cultural Studies of Decision-Making 



The complexity of internal and external environments 
for the contemporary organisations calls for group 
decision making. Most organisations utilise groups, 
such as committees, task forces, teams, and other types 
of groups, rather than individuals, to make important 
decisions. March and Simon (1958), described the 
limitations in the individual’s inherent capabilities of 
comprehending and comparing more than a few 
alternatives at a time as bounded rationality. It seems to 
be reasonable to expect that a group of knowledgeable 
individuals with diverse managerial and technical 
expertise will arrive at more effective decisions than an 
individual. 

One of the reasons for the decision making research to 
be difficult has been claimed by Yates and Lee (1996) to 



be demonstrated in the phrase ‘favourable outcome’. 
What is considered to be as favourable by one national 
culture would not be necessarily considered as such for 
another culture. 

Pious (1993) claimed that what any decision-maker 
‘sees’ as significant can differ markedly from what 
another regards as significant. It is not only the end 
result, the decision outcome, which can be perceived 
differently by different people, even the representation 
of the problem to be decided is different for different 
decision makers. Yates and Lee (1996) define a 
‘representation’ as the decision-maker’s personal 
characterisation of the given situation. It specifies what 
is taken into account by the decision-maker, and what is 
ignored. Representations are based on values, which 
differ in important ways (Schwartz, 1992). For example, 
there are cultures where ‘tradition’ and honouring 
parents and elders are much more important values than . 
‘pleasure’ or an ‘exciting life’. Values work as filters for 
processing the information received and also shape what 
Triandis (1994) called the ‘glasses through which we see 
the world’ or what Pious (1994) called ‘frame of mind’, 
against which different alternatives are judged. 

In different attempts at defining decision making, 
authors have suggested different lists of decision making 
processes (e.g., Boulding, 1975; Janis and Mann, 1977). 
All these processes including ‘representation’ and 
evaluation of different alternatives are readily influenced 
by innumerable conditions, including the abstract 
concept ‘culture’ (Pious, 1994; Yates and Lee, 1996). 

One of the most important limitations of past decision- 
making research is that it lacks a cross-cultural 
orientation. Most decision research is conducted in 
North America and Europe. Relatively few empirical 
research studies have been conducted on how the rest of 
the world makes decisions, and as a consequence, it is 
hard to determine how universal are certain findings 
(Fletcher & Ward, 1988). The following examples 
illustrate the difficulty of drawing firm conclusions: 

• Taylor and Jaggi (1974) found that Hindu Indians 
displayed attribution biases similar to those found in 
the United States, but other researchers have found 
differences in attribution style between Americans 
and Indians. In contrast to Americans who are prone 
to the ‘fundamental’ attribution error, Indians 
attribute behaviour to situational causes more often 
than dispositional causes (Miller, 1984). 
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• Stoner (1968) reported that risky shifts had been 
documented among American, British, and Israeli 
subjects, but other researchers have found 
differences in how American, British, and Asian 
subjects think about probability (Wright et al, 1978; 
Yates et al, 1989). 

• Amir (1984) found that the Asch paradigm produced 
conformity levels in Kuwait comparable to those 
found in the United States, but Matsuda (1985) 
found higher levels of conformity in Japan than in 
United States. Thus findings that appeared general 
one year became culture-bound the next year. 

• Research also suggests that Japanese subjects do not 
exhibit self-serving biases in attribution, unlike 
subjects from the United States, India, South Africa, 
and Yugoslavia (Chandler, Shama, Wolf, & 
Plane hard, 1981). 

• Two recent studies have revealed large differences 
in the apparent decision styles of Chinese and non- 
Chinese; differences that warrant attention. In one 
study, Zhang (1992) made a comparison of 
university students in Mainland China and in the 
United Kingdom. In the other, by Fumham and 
Stringfield (1993), the styles of Hong Kong Chinese 
and European employees of an airline company were 
contrasted. 

• In their experiment, Yates et al (1992) presented 
Taiwanese Chinese, Japanese, and American 
subjects with general knowledge questions. They 
discovered that the Chinese subjects were more 
over-confident in their chosen answers because, 
quite literally, they saw fewer reasons why those 
answers might be wrong. 

In the past two sections the literature on cross-cultural 

studies of values and decision making were reviewed. 

The following points were established: 

1 . In view of the growing globalisation there is a need 
to revisit some of the old theories and examine their 
applicability to different national cultures. These 
would include small group theories and group 
decision-making. 

2. At individual level, Hofstede found Australia to 
have the second highest individualism and Malaysia 
to have the highest power distance. 



THE PRESENT STUDY 
Stage One — A Value Survey 

Two hundred and eight first year undergraduate students 
responded to a 64-question questionnaire. The 64- 
question questionnaire was a self-administered 
instrument, designed to collect data on different cultural 
values. In addition, the questionnaire provided 
demographic characteristics of the population surveyed 
(For a detailed description of the questionnaire and the 
Factor analysis see Rahmati, 1998). All of the useable 
answers to the questions were subjected to factor 
analysis. Factor analysis grouped questions from the 
questionnaire as related to each factor. The name of the 
factors was decided upon based on the types of questions 
factor analysis had associated with each factor. 

In order to investigate differences in value system 
between the two cultures the factors revealed in the 
factor analysis were used to compare the responses by 
national culture to the questionnaires. The mean values 
of the responses to related questions were used to 
calculate an overall mean for each factor. After finding 
the values for factors, their values for the two different 
national cultures involved in this study were compared. 
The statistical test selected to examine the differences 
between the two national groups was the Multivariate 
Analysis of Variance (MANOVA). 

Results 

Some of the nine cultural factors are similar in names 
with those of Hofstede’s but they are different in the type 
of their relevant values. A brief description of each of 
these factors is presented below. 

1. The Religious Commitment Factor: This is the 

relation of an individual to any ideological system, 
in this case a religious system, and it admits of 
different dimensions or types of variation; the 
individual’s acceptance or rejection of the beliefs of 
the system {Orthodoxy), his or her orientation 
toward other persons with respect to his or her 
beliefs {Fanaticism), and the significance of their 
beliefs to their self-conception {Importance) are just 
some of these dimensions. 

2. The Workplace Preferences Factor: The workplace 
items were adopted from the Hofstede ’s Hermes 
study and were actually a part of 14 work goal 
questions which were used by Hofstede to measure 
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for the individualism/collectivism index. The 
response to these questions should show the degree 
of importance associated with each case by the 
participant. 

3. The Locus of Control Factor: The variables in this 
factor are related to the tendency of the individual to 
be in control of his or her own life. The locus of 
control was first developed by Rotter (1966) and 
numerous versions of the original locus of control, 
have been validated and used in different studies 
(although not used in cross-cultural studies). 

4. The Fatalism Factor: The variables in this factor are 
all related to the degree that the individual believes 
that the good or bad in his or her life is predestined, 
and how much the individual believes that every 
occurrence in human existence comes to pass 
because it was fated to do so. 

5. The Traditionalism Factor: The traditionalism 

items, which are related to the fifth factor of the 
overall factor analysis, are related to a kind of 
obedience, which although foreign to Western 
societies, is quite commonly expected in 
traditionalist societies. The traditionalism items are 
related to family (spouse, children and parents) and 
the work relations. 

6. The Challenge and Adventure Factor: The items on 
this factor, which is the sixth factor of the overall 
factor analysis, are more related to readiness to face 
challenge and adventure. Some of these items were 
included in Hofstede’s measure of work goals in 
relation to individualism. The collection of items on 
this factor seems to be related to the individual, not 
necessarily related to work place characteristics. 
The loading of the items on this factor are more 
towards a belief by the individual in personal 
competence. 

7. The Individualism Factor: The items on this factor 
are related to individualism. Some of the items are 
symbols of Competitive Individualism as defined by 
Triandis et al (1993). There are also items on this 
factor related to what Triandis et al define as 
‘Independence’ as part of individualism. 

8. The Value of Privacy Factor: The three items in this 
factor are very much related to the individualism 
value for privacy and the items are all positively 
related to the value of privacy factor. The value of 



privacy items are related to the individual’s need for 
personal time and a personal set of friends in family 
life, and a valuing of the private opinion of the 
individual in the work place. 

9. The Uncertainty Avoidance Factor: This factor is 
related to the different methods used by individuals 
to avoid uncertainty and to reduce the risk of 
unforeseen. Three items are related to sharing the 
risk with friends, and one item is related to choosing 
a large corporation over smaller ones to avoid the 
risk of losing a job. The last item is related to 
planning as yet another way to avoid the uncertainty 
of the future. 

The multivariate analysis of variance for the above 
factors with nationality as the independent variable 
revealed a significant multivariate effect for nationality. 

DIAGRAM 3 

THE COMPARISON OF THE 
SCORES CALCULATED FOR THE 
TWO NATIONAL CULTURES BASED 
ON THE NEW 9-FACTOR MODEL 




Stage Two — Laboratory Quasi-experiments 

Triandis (1994) mentioned the difficulty of ensuring that 
the same degree of manipulation of the independent 
variables has been used in each setting involving a 
cultural study. As a result it was decided to conduct the 
study using the same setting, that is to complete different 
parts of the study involving the two sets of national 
cultures in Australia instead of conducting the study in 
the two different countries involved. 

The focus of this part of the study is on the value system 
referred to by the participants from the two nationalities 
in working with only one task used in groups. The 
details of the task and the qualitative analysis of the data 
can be seen in Rahmati (1998). 
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The groupwork of 95 participants, 50 Malaysian and 45 
Australian participants was accepted for the purpose of 
this research. With the exception of one Australian 
group of four, all other groups had five members. 

The resulting data from groupwork was a transcribed 
text of the tape recordings of the discussions of group 
sessions. This data was subjected to content analysis. 

The method of content analysis followed an approach 
suggested by Morse and Field (1995). The researcher 
reads the entire document and identifies several of what 
they call 'topics'. These topics become the main 
categories or category labels. According to Berg (1995), 
the categories should be very broad at first to allow a 
large number of comments to be grouped into each 
category. Later these categories are divided into 
subcategories as the need arises. As different classes of 
comments (topics) emerged from the comments a 
diagram showing the relationship between different 
categories and subcategories in the data was developed. 
On the basis of the classification of categories in this 
diagram the codes were allocated to different classes of 
comments. The next step was to enter the data file and 
allocate the codes and develop the tree (the classification 
of comments) into Q.S.R. Nudist. The topics were 
selected based on their meaning and could be a part of a 
sentence, a complete sentence or even a paragraph, 
which would be allocated a code. The code would guide 
the software as to where the place of that special theme 
is compared to the rest of the tree. 

Results 

It was to be anticipated that not the values relevant to all 
factors would be referred to by groups of the two 
national cultures in each mode of decision-making 
situation. For the single task used in this study the 
participants referred to values relevant to 6 factors out of 
nine: Religious Commitment, Traditionalism, Locus of 
Control, Challenge and Adventure, and Fatalism. While 
the values of these six factors have been at work for this 
single task but the degree of their reference is different 
for different national groups. 

There are some values related to some of the factors 
referred to by one set of national group that are missing 
from the list of factors referred to by the other one. 
These values and their relation to the factors are as 
follows: 



Diagram 4 compares the reference made to the six 
cultural factors (Y-axis) by groups of the two national 
cultures. It should be noted that X-axis in this Diagram 
illustrates the relative frequency (percentage) of the 
groups, which have referred to the values of a factor. 
The diagram shows that no reference was made to any 
values relevant to Traditionalism, Challenge & 
Adventure, and Religious Commitment Factors by 
Australian participants. Malaysian participants made no 
reference to any values relevant to Internal Locus of 
Control. 



Duty to God 

Duty towards Country' & Society 

Duty towards family 

Status 

The ability to make plans work 
Willingness to go to faraway places 
Tlic belief in lucli and fate 



Religious Commitment 

^ Traditionalism 

Locus of Control 
Challenge & Adventure 
Fatalism 

Uncertainty Avoidence 



The willingness to share the responsibility 
for anv decision 



DIAGRAM 4 

THE COMPARISON BETWEEN THE 
CULTURAL VALUES REFERRED TO BY 
GROUPS FROM AUSTRALIA AND MALAYSIA 




■ Malayilsn 
□Auatratian 



DISCUSSION 

It was established that, facing the same decision-making 
problem in a similar decision-making environment, 
groups of the two different national cultures referred to 
different cultural factors and, based on these in some 
cases reached different decisions. It was also discovered 
that, in the cases where the two sets of national groups 
made similar decisions, their selection was based on 
different values. 

The Australian participants who had high individualism, 
low traditionalism, and high internal locus of control 
were a clear example of a ‘loose’ society. According to 
Pelto (1968) and Triandis (1994) in a loose society the 
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values relating to the formalisation of group behaviour 
are undeveloped because deviant behaviour is easily 
tolerated. 

The comments made by Malaysian groups showed 
significantly higher uncertainty avoidance, religious 
commitment, and collectivism, than the Australian 
participants, as well as high traditionalism and external 
locus of control (Fatalism). Such a set of values would 
define Triandis’s (1994) ‘tight’ society. The detailed 
cultural characteristics of groups from the two national 
cultures are examined below. 

1 . The high score for the Uncertainty Avoidance factor 
for the Malaysian students, found in this study, 
indicated a willingness to share the decision-making 
process in order to decrease the risks involved. The 
reason for this interpretation is that three of the five 
items of the Uncertainty factor are related to sharing 
the responsibility in a decision-making situation. 
Australian students showed considerably lower 
uncertainty avoidance, which is close to the findings 
of their value survey. 

2. Triandis (1994) believed that groups in a collectivist 
society are the basic units of social perception. The 
high collectivism (or the low individualism) of the 
Malaysian participants would encourage group 
decision-making in any decision-making situation. 
A very low collectivism is a confirmation of the 
presence of high individualism for Australian 
students who believe that they are responsible for 
their own life. 

3. The high traditionalism factor for the Malaysian 
groups suggested the existence of a set of social 
regulations for influencing any type of social 
interaction, including group interactions. This set of 
regulations includes a respect for seniors and people 
of high status in the group. Such social regulations 
could be considered as a restriction in terms of what 
would be considered proper behaviour by group 
members in aspects of group decision-making, such 
as turn taking, and willingness to disagree with the 
suggested alternative of their seniors. In Australia as 
a loose society, individuals are accepted for their 
own personal merits and not because of their 
seniority or the class or their family. There may be 
little reason for individuals as group members in 
such a society to hide their disagreement in a 
decision-making situation. 



4. The strong rating accorded to an external locus of 
control (Fatalism) by the Malaysian participants 
suggests they may be more likely to engage in risk 
taking behaviour (Battle & Rotter, 1963), because 
there is a belief that any consequences would be the 
responsibility of the superior external forces. This 
contrasts with the high internal locus of control 
shown by Australian groups. With high internal 
locus of control the decision-makers believe that 
they are responsible for the consequences of their 
decision. As a result to reduce the risk of a bad 
decision, the decision-making groups discuss all 
aspects of each alternative in detail before making 
their final decision. In such situation group 
members need their own air time to discuss their 
points of view. 

The sum of the above features results in a situation for 
the Malaysian participants in which, although 
collectivism and religious commitment encourage the 
tendency towards group decision-making, the adherence 
to high traditionalism results in some limitations in 
group interactions. Malaysian participants are likely to 
be significantly limited by their social rules concerning 
group interactions. These limitations may include being 
obliged to agree with their seniors and refraining from 
expressing their opinion in group sessions. 
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ABSTRACT 

On April 2, 2000, Georgia Southern University’s newly inaugurated President announced the creation of a School 
or College of Information Technology as a facet of the Governor ’s rural economic development initiatives. The new 
academ ic unit, scheduled to be implemented FallSemester 2001 , is tasked with at least tripling the university ’s annual 
production of IT undergraduates and will be housed in a new building. This paper will describe the steps taken to 
operationalize the newly announced academic unit with special emphasis on curricular issues. It summarizes key 
aspects of a white paper produced by an ad hoc College of Business Administration committee. The white paper 
documents the need for the new academic unit, identifies targeted industries, outlines a forward-thinking curriculum 
model, proposes degree programs, includes student and faculty projections, summarizes major facilities and resources 
needs, proposes the creation of an ''elnstitute, ’’and provides an overview of a preliminary marketing/promotion plan. 
As such, the whitepaper ’s format and content are valuable to other universities faced with the challenge/opportunity 
to rapidly expand their IS/IT programs. 
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INTRODUCTION 

On Sunday, April 2, 2000, Dr. Bruce Grube announced 
the creation of a school/college of Information 
Technology during his inaugural address as Georgia 
Southern University’s newest President. He noted that 
the new academic unit was part of Georgia Governor 
Ray Barnes’ “One Georgia (rural economic 
development). Dr. Grube also stated that $500,000 in 
planning money for the new school/college has been 
included in the state legislature’s supplemental budget 
and that a new building to house the academic unit’s 
programs had been placed on the Board of Regent’s 
“super-fasMrack construction list. 

The rationale for this initiative began at the Governor’s 
office. Georgia Governor Ray Barnes wants to create a 
statewide environment that will enable the New 
Economy to flourish. He is widely viewed among 
movers and shakers in the state’s IT professional 
community as a governor who “gets it. For example, 
soon after taking office in 1999, Governor Barnes 
announced the Yamacraw Mission aimed at attracting 
high bandwidth communications expertise, companies 
and jobs to the state. Because an appropriately skilled 
workforce is essential for enticing high-tech firms to 
relocate or expand to the state, a key facet of the 
Yamacraw Mission is an increase of more than 40 
faculty members in targeted disciplines at the Georgia 
Institute of Technology. The occupants of these new 
positions are tasked with significantly increasing the 
University System of Georgia’s (USG) annual 
production of graduates with the skills needed by high 
bandwidth communications companies. Some of these 
faculty members would actually teach at other USG 
institutions through the Georgia Tech Regional 
Engineering Program (GTREP). Through GTREP, 
Georgia Southern University is home to Southeast 
Georgia’s first computer engineering degree program. 

Heeding private sector advice that the state should lead 
by example if it wants to attract e-business firms to the 
state. Governor Barnes has also promoted e-commerce 
applicationsamong state agencies. In 1 999, Georgia was 
recognized as leading the nation in government e- 
commerce applications. 

The governor has also commissioned a number of 
studies to pinpoint IT work force needs across the state. 
In one report, E-Commerce and the State of Georgia, 
McLean, McDonald, and Case (1 999) indicated that E- 
commerce activity was much more vigorous in the 



Atlanta rnetropolitan area than in most otherparts ofthe 
state. The report recommended concerted efforts to 
leverage existing private sector and USG infrastructure 
strengths in other bright spots in the state including 
Columbus and Savannah. The creation of a 
school/college of Information Technology at Georgia 
Southern that is tasked with at least tripling it annual 
undergraduate output of IT majors will make the coastal 
region of Georgia (and the state as a whole)an attractive 
site for high-tech relocation and expansion. 

Initial Reactions 

Three days after the public announcement of the new 
school/collegeof IT, Carl Gooding, Dean of the College 
of Business administration met with the faculty of the 
Department of Information Systems and Logistics. The 
purpose of the meeting was to describe the genesis ofthe 
new initiative, his understanding of the process of that 
would be taking place at the university level, and his 
recommendations of for how the department andcollege 
should respond. Dean Gooding confirmed that this was 
a top-down initiative that originated at the governor’s 
office. He noted that the Dean of the College of 
Technology had already drafted a short “white paper 
callingfor the establishment of a Collegeof Information 
Technology; he also noted that a preliminary program 
statement for the new building had been developed by 
another administrator. More importantly, the dean stated 
that President Grube felt that the College of Business 
Administration should drive the creation and 
implementation of the new academic unit even though 
degree programs outside the College of Business (such 
as computer science andgeographicinformation systems 
and possibly computer engineering) would probably be 
directly or virtually related to the new school/college. 
Dean Gooding noted that the President had encouraged 
him to “be aggressive and that a S chool of IT within the 
College of Business seemed to be an appropriate first 
step toward what one day might evolve into an 
independent College ofinformation Technology. 

Creation of the COBA Committee 

At the meeting with the Department of Information 
Systemsand Logistics, Dean Gooding announced that he 
would be appointing an ad hoc COBA committee 
composed primarily of IS faculty and representatives of 
other IT-oriented business disciplines including 
accounting information systems, logistics, and 
operations management. The director of the Center for 
Econom ic Development would be on the committee and 
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representatives from other disciplines (including 
computer science and geography) would be invited to 
participate. The committee would be tasked to: a) 
immediately develop a “white paper summarizing 
COBA’s recommendations for a School of Information 
Technology, and b) work with the soon to be appointed 
university-level steering committee on anongoing basis 
to ensure that the new academic unit degree programs 
have an appropriate business and econom ic development 
orientation. 

Over the next two days, the COBA committee was 
announced, charged, and began meeting several times a 
week. One of its first actions was to create a “white 
paper sub-committee composed of IS faculty members. 
In order to have a draft of a COBA white paper before an 
April 29 Business Advisory Council meeting, and befo re 
the appointment of the university-level committee, this 
committee began meeting on a daily basis. 

The White Paper Sub-Committee 

Because COBA would have less than fifty percentof the 
seats on the university-level committee for the 
school/collegeof IT, wordingin the white paperbecame 
a critical issues. While desiring to develop compelling 
arguments for establishing a School of Information 
Technology within COBA, the sub-committee 
recognized the value of being inclusive. 

This sub-committee decided thatone way to accomplish 
this was to maintain focus on the economic development 
aspects of the new academic unit. Emphasis would be 
placed on creating a readily available work force for 
attracting high technology firms to Southeast Georgia. 

The sub-committee also chose to focus on the rapid 
evolution of IT is rapidly evolving and the difficulties 
that can be encountered when trying to delineate its 
boundaries. In order to connect readers to disciplines 
already in place at Georgia Southern, the white paper 
sub-committee chose to conceptualize IT as being 
grounded in the three disciplines depicted below. 

The strength of an academic program in Information 
Technology accrues from the blending of concepts from 
these three areas. Traditionally, computer science 
programs provide rigorous grounding in computing 
theory, programming techniques, and algorithmic 
processes. Computer engineering programs address the 
research and development of digital circuitry andrelated 
systems that comprise the infrastructure of any 
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computing environment. Information systemsprograms 
are more applied, emphasizing the integration of 
fundamental programm ing principles, network 
administration, and database design to create business 
applications. 



InfontiitHon Technology 
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Across all three disciplines there is a need to address 
modern applicationsof IT which are increasingly reliant 
on the development and support of business strategies 
based on eCommerce. This may be illustrated in the 
following manner. 




To ensure inclusiveness, the white papersub-committee 
chose to define eCommerce in a very broad sense that 
draws from all areas of information technology. E- 
Commerce’s primary goal is to meet the needs of 
business, government, and nonprofit organizations 
within an electronicenvironment. As a result, necessary 
theory should be integrated with the rigorous 
developmentof applied IT skills. These programs should 
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address specific industry needs and support the overall 
economic health of the region, state, and nation. 

DOCUMENTING THE NEED 
FOR MORE IT PROFESSIONALS 

The sub-committee turn edto several sources to establish 
the need more appropriately skilled IT workers in the 
region, state, and nation. Compelling evidence for 
national IT worker shortages came from a recent report 
issued by the Information Technology Association of 
America (ITAA ) indicating th at 1 in 1 2 IS/IT jobs in the 
U.S. are currently vacant. This report also indicates that 
more than one half of the 1 .6 million new IT jobscreated 
this year are likely to go unfilled (see: http^/www.itaa. 
org/workforce/studies/hwOOexecsumm .htm). The ITAA 
findingsare consistentwith those inreports issued by the 
U.S. Department of Commerce (e.g.. The Digital Work 
Force: Building Infotech Skills at the Speed of 
Innovation, July 1999). 

Further support for IT work force increases at the 
national level is provided by The Emerging Digital 
Economy ( http://www.commerce.gov/ede/report.html), 
the Department of Commerce predicts that by 2006, 
almost one half, or 49 percent, of the private work force 
will be employed either by IT-producing companies 
(manufacture rs/providers of IT equipment or services) or 



firms that are heavy users if IT. In 2006, the work force 
of IT-producers is expected to exceed 6 million while the 
work force of heavy IT-users will grow to more than 51 
million. Annual work force increases in IT-producing 
industriesare expected to range between 7 and 8 percent 
between now and 2006. Similar annual increases are 
expected in IS/IT positions in heavy IT-using firms. In 
IT-producing industries, software and services are 
generating the largest demand for new workers; 
significantly smaller gains are expected in firms that 
manufacture computer hardware or communications 
equipment. 

The sub-committee also summarized the findings of 
studies indicating rapid growth in Georgia’s IT work 
force as well as growing IS/IT worker shortages. For 
QxamplQ, Electronic Commerce and the State of G eorgia 
(www.icapp.org/pubs/index.html) reports that more than 
5,500 new “core IT jobs, i.e., computer scientists, 
computer engineers, systems engineers, computer 
programmers, will be needed annually in the state until 
2006. This is presented in detail in the following table. 

McLean and McDonald (2000) and Chaker (2000) also 
report that Georgia leads the Southeast in the growth of 
IT workers and is second nationally in such growth. The 
following table illustrates this growth. 



CORE IT JOBS IN GEORGIA, 1996-2006 



Occupational Title 


1996 

Employment 


2006 

Employment 


Total Change 


Percent 

Change 


Average Annual 
Openings 


Computer Scientists 


6,000 


16,200 


10,200 


171 


1,080 


Computer Engineers 


4,400 


1 1,300 


6,900 


156 


720 


Systems Analysts 


13,600 


34,300 


20,700 


152 


2160 


Computer Program mers 


18,750 


28,950 


10,200 


54 


1,600 


Totals 


42,750 


90,750 


48,050 


1 12 


5,560 



(Source: The Digital Workforce, U.S. Department of Commerce, July 1999) 



IMPORTANT IT LABOR MARKET STATISTICS FOR STATES IN THE SOUTH AND SOUTHEAST 



State 


No. of IT Workers 
Employed (Estimate for 
2006) 


Ranks 


Fastest Growth 
% Increase 
1996-2006 


Ranks 


Average Annual 
Job Openings 
1996-2006 


Ranks 






Region 


US 




Region 


US 




Region 


US 


Alabama 


31,000 


4 


19 


64 


3 


20 


1,730 


4 


18 


Florida 


95,400 


1 


7 


63 


4 


23 


4,620 


2 


9 


Georgia 


90,750 


2 


9 


112 


1 


2 


5,560 


1 


5 


Mississippi 


7,350 


7 


33 


54 


7 


35 


330 


7 


33 


North Carolina 


69,950 


3 


12 


80 


2 


7 


3,770 


3 


12 


South Carolina 


16,250 


6 


23 


56 


6 


31 


770 


6 


23 


Tennessee 


27,550 


5 


20 


59 


5 


28 


1,310 


5 


19 



(Source: The Digital Workforce, U.S. Department of Commerce, July 1999) 
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Several other studies were used by the white paper sub- 
committee to reinforce the need for the new academic 
unit at Georgia Southern. For example, Burriss (1999) 
summarized the results of a indicating that as many as 1 
in every 3 new IT positions in Georgia will go unfilled 
if steps are not taken by the state to increase the supply 
of appropriately skilled workers. If these needs are not 
addressed, the growth of Georgia’s high technology 
sector will be slowed as will the growth of businesses 
that rely on IT. 

Burriss (1999) also notes that while the University 
System of Georgia (the state’s largest source of IS/IT 
graduates) has increased the number of degrees 
conferredeachyearof computer programmers, computer 
engineers, and systems analysts from 1996 to 1998, a 
tremendous gap between supply and demand still 
remains. Currently there are annual shortfalls of more 
than 1,350 in just these three IS/IT occupations. Total 
shortfalls will mushroom to more than 13,000 by 2005 
if USG does not increase graduates in these areas. 

In sum, data from numerous sources was used by the 
white paper sub-committee to document the growing 
demand for IS/IT workers as well as growing shortages 
of appropriately skilled workers. The findings suggest 
that the USG and Georgia Southern University will be 
better positioned to address regional, state, and national 
needs for IS/IT workers through the implementation of 
this new academic unit. The data also suggest that the 
creation of this academic unit will strengthen southeast 
Georgia’s ability to attract firms that are producers of IT 
and firms that are heavy users of IT to this region. This 
is consistent with the rural economic development 
initiatives associated withthe governor’s “One Georgia 
initiative. 

Targeted Industries 

An important aspect of Governor Barnes’ economic 
development initiative is focused on attracting and/or 
facilitating the expansion of firms that may be classified 
as IT-Producers or Heavy IT-Users. Examples of firms 
falling into these two categories are illustrated below: 

IT Producing Organizations 

Hardware Manufacturers 

e.g., Dell, Intel, Sun, etc. 

Software Developers 

e.g., Microsoft, Oracle, SAP, JD Edwards, etc. 



IT Infrastructure Developers 

e.g., AT&T, MCI/WorldCom, Cisco, UUNET, 
etc. 

Consulting Firms 

e.g., IBM Global Services, EDS, answer/Z^iwA:, 
KPMG, Anderson Consulting, etc. 

Heavy IT User Organizations 

Retail 

e.g., W al-M art, The Ho me D epot. Office D epot, 
Amazon.com, etc. 

Service 

e.g., UPS, Southern Company, BellSouth, 
American Express, Delta, AFLAC, 
Synovus, health care, etc. 

Manufacturing 

e.g., Coca-Cola, Ford, GM, Caterpillar, Briggs 
& Stratton, etc. 

Firms that fall into these distinct categories will have 
unique requirements and needs for IT graduates. In 
order to support the governor’s economic development 
initiatives, the COBA committee felt that the School of 
Information Technology should have a curriculum 
capable of meeting as many of these diverse needs as 
possible. Toward this end, the COBA committee 
recommended the solicitation of direct input from 
multiple firms in each category as well as the creation of 
an advisory board comprised of representatives fi'om 
firms in each category to guide, direct, and update the 
curriculum. The COBA committee also recommended 
soliciting ongoing input from Georgia Southern 
University’s alumni who work in the IT field. 

CONCEPTUAL MODEL FOR A 
CURRICULUM TO ADDRESS THE WORK 
FORCE NEEDS OF TARGETED INDUSTRIES 

In order to develop a sufficiently diverse curriculum to 
meet the work force needs of targeted industries, 
COBA’s white paper subcommittee recommended a 
conceptual curriculum model composed of an 
eCommerce core and multiple sub-discipline 
specializations or 'pods'. Each of these is considered to 
be dynamic and is expected to continue to evolve. The 
initial conceptual model calls for five distinct “pods 
designed to meet specific skill shortages in the targeted 
industries. These pods are illustrated and discussed 
more fully below: 
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requiring systems consultants, implementers, trainers, 
and problem resolution specialists. 




Systems Development and Support 

The Systems D evelopmen t and Support pod is proposed 
to address the business and industry need to develop and 
deliver core business applications, i.e,, accounting, 
manufacturing, marketing, logistics, etc. This pod 
includes the development of applications using both 
traditional and object-oriented (a new programming 
methodology in which objects are builtand manipulated) 
approaches. It will support legacy (old, traditional) 
systemsas well as current online transaction processing 
(OLTP) systems. Integrated into this sub-discipline will 
be fundamentals of languages, systems analysis and 
design, data management, and data communications. 
Graduates will be prepared forpositionswith companies 
where IT is the primary product or service the company 
is generating as well as with companies that are 
themselves heavy users of IT products and services. 

Information Technology and Business Systems 
Integration 

The proposed Information Technology and Business 
Systems Integration pod is designed to address the 
business and industry need to deliverintegrated systems 
solutions. Primarily focused on consulting, implemen- 
tation, training, and support, this pod will include the 
installation and integration of packaged (or previously 
developed) business applications solutions, i.e,, 
Enterprise Resource Planning(ERP) systems, Customer 
Relationship Management (CRM) systems, and Supply 
Chain Management (SCM) systems . This pod will focus 
on systems analysis and design, sy stemsimplementation, 
project management, and decision analysis. 
Fundamental concepts of languages, data management, 
and data communications will be incorporated into this 
area of study. Graduates will be prepared for positions 
with companies that deliver IT products and services 



Web Development and Support 

The Web Development and Support pod is designed to 
address the business and industry need to develop and 
deliver web-based applications. Primarily targeting 
web-based and web-enabled systems, this pod will 
include the development of applicationsusing an object- 
oriented approach with web languages (i,e,, Java, 
HTML, DHTML, XML, Perl), data management (CGI, 
database interface/integration), and network 
management. Key areas of emphasis include user 
interface design, graphics design, and electronic 
publishing. Fundamental concepts of systems analysis 
and design, systems implementation, and project 
management will be included in this area of study. 
Graduates will be prepared forpositionswith companies 
in which the IT product or service requires web 
development and support skills, as well as with 
companies that are heavy users ofweb-based IT products 
and services. 

Telecomm unications/Network Specialist 

The Telecommunications/Network Specialist area of 
study will address the business and industry needs to 
develop and support telecommunications networks. This 
pod will include the design, development, imple- 
mentation, and management oforganizational networks, 
including local area networks (LAN) and wide area 
networks (WAN). Key areas of emphasis will be user 
network analysis and design, internet/intranet/extranet 
developm ent, electronic data interchange (EDI), and net- 
work managem ent. Fundamental concepts of languages, 
data management, and project management will be 
included. Graduates will be prepared for positions with 
companies that offer an IT product or service which 
requires telecommunications networks and with 
companies that are heavy users of IT telecom- 
munications network products and services. 

Knowledge Management and Analysis Specialist 

The Knowledge Management and Analysis Specialist 
pod will address the business and industry need to 
design, implement, and analyze enterprise (collected 
from the entire organization) data. This pod will include 
the design, development, implementation, and 
management of enterprise data warehouses and vision 
based analytical systems, i.e., geographic information 
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systems (GIS). Key areas of emphasis would be 
decision analysis, online analytical processing (OLAP), 
data mining, and spatial data modeling and analysis. 
Fundamental concepts of data management, database, 
and programming languages will be included. These 
graduates will be soughtby companies in which IT is the 
product or service as well as with companies that are 
heavy users of IT products and services requiring intense 
data collection, storage, and analysis with a focus on 
improving organizational decision making. 



Proposed Degree Models 

Because the School of Information Technology is on a 
fast-track implementation schedule,^ the COBA 
committee recommended leveraging existing IS courses 
as much as possible in the new academic unit’s 
curriculum. The current BBA/I S curriculum at Georgia 
Southern includes the following IS courses: 



CISM2130 
CSCI 1236 
CISM3130 
CISM 3134 

☆ CISM 3135 

☆ CISM 4134 

☆ CISM 4135 
CISM 4136 



Computers and Applications 
Introduction to Java 
ApplicationDevelopment (in Java) 
Business Data Communications 
Systems Development 
Database Applications 
Project Management 
Global Information Resource 



Management 

'Cc Two CISM electives (most commonly CISM 
4236 Decision Support Systems; CISM 4247 
Object Oriented Programming; CISM 4238 
Network Administration) 



The current curriculum provides also hands-on 
experience with a variety of languages and software 
packages including HTML, Java, Visual Basic, MS 
Office, MS Project, Designer 2000, Developer 2000, 
NefW are 5, W indows 95/98 and W indows 2000. 



The COBA committee also recommended that all degree 
programs in the School of IT should include at least a 
minor in business. This would help to ensure that 
graduates of the School of IT possess an appropriate 
grounding in business in addition to a technical focus. 



In response, the white paper sub-committee proposed to 
ensure a technical focus by requiring majors in any 
COBA-controlled degree program to take multiple 
courses in at least one of the pods. This committee also 



proposed two COBA-controlled degree options within 
the School of Information Technology. Both would 
leverage existing business and IS courses and would 
achieve the appropriate technical focus needed by 
targetedindustries via courses in one or more ofthe pods 
identified in the conceptual model. 



The BBA/I S degree would co ntinue to provide stu dents 
with a strong business orientation, but would enable 
students to gain additional technical skills associated 
with a particularpod. The proposed BS/IT degree would 
allow students to pursue a business-oriented degree 
while gaining technical skills associated with two pods. 
The two degree programs are outlined below: 



BBA in 

Information Systems 

8 courses constitute 
BBA core 

7 courses in major: 

4 courses constitute 
‘IS core’ 

1 pod, consisting of 
3 courses 



Bachelo r of Scien ce in 
IT 

4 courses constitute 
business minor 

) 

12 courses in major: 

6 courses constitute 
‘IT core’ 

2 pods, each 

consisting of 3 
courses 



Other degreeprograms within the School ofinformation 
could include current or proposed degrees in Applied 
Computer Science and/or Com puter Engineering. If the 
new academ ic unit is housed w ithin COB A, the stude nts 
enrolled in these degree programs would be encouraged 
(but not required) to pursue a minor in business. Only 
students enrolled in COBA-controlled degreeprograms 
(the BBA/IS and proposed BS/IT) within the School of 
Information Technology would be required to obtain at 
least a minor in business. 

OTHER KEY ELEMENTS IN THE 
COBA WHITE PAPER 



Beyond documenting the need for a School of IT and 
outlining a forward-looking curriculum model and 
degree programs, the COBA white paper included 
several key elements. The key elements include 
projected student enrollment, projected faculty needs, 
facility requirements, a proposal to create an elnstitute, 
and a preliminary marketing and recruiting plan. These 
are discussed in the following sections. 
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Student and Faculty Projections for the School of 
Information Technology 

A stated goal ofthe School of IT initiative is to produce 
300 graduates per year by 2005. The numbers of 
degrees granted from the Information Systems (IS) and 
Computer Science (CS) programs at GSU for the last 
four years are shown below. Together these programs 
will generate approxim ately 100 graduates this academic 
^ year. Historically, the mix of graduates from these 
programs is about 75 percent from IS and 25 percent 
from CS. Both programs have experienced steady 
growth, in spite ofthe “anomaly created by conversion 
to the semester system just prior to the 1998-1999 
academic year. This change inspired many students in 
both disciplinesto complete degree requirements prior to 
the conversion, thus creating a higher than normal 
number of graduates in 1997-1998 and a lower than 
normal number in 1998-1999. 



Student Projections 

Assuming the historical 75/25 mix continues, it is 
estimated that the BBA/IS and the proposed BS/IT 
programs would account for 200+ of the targeted 300 
School of IT graduates in 2005, with the remaining 
graduates coming from other applied degree programs. 
To produce these 200 graduatesannually,the number of 
Junior/senior majors in the BBA/IS an d proposed B S/IT 
degree programs would need to be double that number 
(or approximately 400). The historical growth rate for 
junior/senior IS majors is approximately 20 percent per 
year. If this growth rate continues and is supplemented 
by aggressive student recruiting programs, the projected 
the number of junior/senior majors and degrees granted 
from the proposed COB A programs to be as follows: 




Projected COBA Majors & Degrees 




|0Jr/Sr Majors HDe^eesj 



o 
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Junior and Senior Enrollment in BBA/IS and proposed BS/IT programs 



Year 


1999/2000 


2000/2001 


2001/2002 


2002/2003 


2003/2004 


2004/2005 


Majors 


155 


200 


250 


300 


350 


425 


Degrees 


75 


90 


110 


135 


160 


195 



Faculty Projections 

In order to accommodate the projected number of 
junior/senior BBA/IS and BS/IT students, more than 60 
sectionsof required and elective courses will be required 
each semester by the 2004/2005 academic year to 
support anticipated majors. If each faculty member 
teaches 2 upper level IS/IT courses at 25 students per 
course with the balanceof the teaching load per semester 
being service courses at 40 students per course, a 
minimum of 8,900 credit hours will be generated. If 
each faculty member is responsible for 280 credit hours 
per semester, as recommended by the AASCB, 32 
faculty members will be needed to by the 2004/2005 
academic year to deliver the BBA/IS and the BS/IT 
programs. 

The following table provides targets for COB A IS 
faculty needs for the BBA/IS and proposed BS/IT 
programs at the end of each academic year. Additional 
faculty would be required for other programs. 

FACILITIES NEEDS 

Student and faculty projections clearly indicate that a 
building will be required to accommodate School of 
Information Technology degree programs. The following 
issues were considered in the preliminary planning for 
this new facility. 

Space Considerations/Assumptions 

It is anticipated that all BBA/IS and BS/IT courses and 
labs will be taught in thenew building. It is also assumed 
thatthe core/service IS courses will continueto be taught 
in the COBA building and will utilize existing three 
COB A labs (which are currently capable of handling, 33, 
42, and 58 students). It is assumed that this new building 
will provide adequate classroom, administrative 
staff/suppo rt, and faculty office space to meet the student 
and faculty projections. 



Desired Technology Infrastructure Features 

A high bandwidth fiber LAN should be installed in the 
new building. In anticipation of IT students being 
required to own laptop computers, connections to the 
network are recommended for each classroom seat and 
student work space (including multiple student 
work/meeting rooms). 

Computing Labs and Resources 

If School of IT students are required to own laptops, 
much of the need for traditional computer lab will be 
eliminated. However, the nature of the proposed IT 
curriculum will require several specialized labs to 
provide experience with specialized hardware and 
software, including the following: 

1. A computer lab to provide experience with 
networking hardware and software. 

2. A computer lab to provide experience with web 
development tools including advanced 
scanning/digitizing hardware and graphics 
editing/design software. 

3. A computer lab to provide experience with virtual 
reality/3D modeling and data analysis. 

4. A computer lab for hands-on experience with 
collaborative computing. 

Multiple will be required to provide operational support 
of the school. In addition, multiple servers will be 
required for dedicated, educational support (e.g., 
database driven web development, SAP, etc.). 

Classroom Features 

To attract high-quality IT students, each classroom in the 
School of Information Technology should be equipped 
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COBA IS FACULTY NEEDS 



Year-^ 


1999/2000 


2000/2001 


2001/2002 


2002/2003 


2003/2004 


2004/2005 


Majors 


155 


200 


250 


300 


350 


425 


BBA/IS 


155 


200 


200 


225 


250 


300 


BS/IT 


0 


0 


50 


75 


100 


125 


Faculty 


10 


18 


22 


25 


28 


32 



state-of-the-art educational technologies. These tech- 
nologies will include network connections for each 
student’s laptop, computer/video projection capabilities, 
the ability to project any student’s screen to the class, 
VCR/DVD player and audio system, an overhead 
projector, a built in computer or computer hookup for 
instructor, and distance learning broadcast capabilities. 

elnstitute Proposal 

The COBA white paper sub-committee proposed the 
development of an “elnstitute as an integral component 
of the School of Information Technology. The proposed 
institute will provide a mechanism to stimulate economic 
growth in South Georgia and provide services that 
extend beyond the academic degree programs. It will 
work cooperatively with both internal and external 
constituents, i.e., GSU departments, other university 
system schools/colleges, area technical schools, 
businesses, and other independent entities, to meet the 
challenge of preparing South Georgia to emerge as a 
leader in the high technology arena. 

The elnstitute will offer certificate programs in 
information systems and information technology to 
prepare graduates in other disciplines for entry level 
positions in IT. The certificate programs will be 
designed to be completed in one year (or less) and offer 
a “fast track solution to meeting the growing demand 
for IT workers. 

Another key element of the proposed elnstitute is an 
eBusiness incubator. The incubator will provide 
facilities and resources for high tech start-up firms to 
develop and mature. It will involve faculty and students 
as a pool of available expertise to assist these young 
companies as they evolve. The incubator will work 
closely with Georgia Southern’s new College of 
Economic Development in promoting the economic 
growth of South Georgia. 

To partially address the need for additional faculty 
resources, the elnstitute will develop an ICAPP 
(Intellectual Capital Assistance Partnership Program) 



Faculty Development Program, Like the now defunct 
AACSB MIS Faculty Development Institutes, this 
ICAPP program will train current terminally qualified 
faculty in other disciplines to meet the needs of the 
School of Information Technology as well as the need 
forlS/IT faculty at other University System of Georgia 
institutions. 

Development and Promotion Planning 

The final key elementin COBA ’s School of Information 
Technology White Paper is a preliminary marketing and 
recruiting plan. In order to attract high quality majors 
and faculty, the university must comm it the resources to 
effective marketing the new academic unit. Some of the 
suggestions included in the preliminary plan include: 

General Marking Recommendations 

1. Announcing the program via press releases, formal 
announcement cards, advertisements in professional 
publications, etc. 

2. Sponsoringreceptions and/or coffeesat professional 
meetings to announce the program and recruit 
faculty. 

3. Developingbrochures and other marketing materials 
to distribute to high schools, junior colleges, and 
other sources of prospective students. 

4. Offeringto speak to professionalorganizations, civic 
groups, and alumni groups. 

5. Participating in university student recruitment 
receptions. 

Student Recruitment Recommendations 

1 . Work with the admissions officeto secure “Potential 
Student List(s) of high school students who have 
both the background and skills for success in a 
strong IT program. 
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2. Design brochures, letters, etc. that are specially 
targeted to high-ability high school and junior 
college students. 

3. A mail/phonerecruitmentplan utilizing professional 
staff, faculty, current students, and alumni. 

4. Faculty, student, and alumni attendance at 
recruiting/promotion events. 

5. Targeted recruiting efforts designed to attract 
individuals already in the workforce who want to be 
retrained in I S/IT. 

6. Active recruitment of high ability undeclared 
majors at GSU. 

7. Emphasize the availability of HOPE, University 
Honors Programs, Bell Honors Program, and IS 
scholarships. 

8. Initiate a summer “IT camp for high ability high 
school students. 

9. Sponsor IT competitions for high school students 
during the academic year. 

10. Create an IT Scholars Program for high school 
juniors and seniors. 

11. Develop a recognition program for Georgia high 
school students scoring a 4 or a 5 on the AP 
Computer Science Exam. 

12. Directly recruit at IT-oriented magnet schools. 

Faculty Recruiting Recommendations 

1. Program announcements in all appropriate 
professional publications/placement materials 
including IS journals. 

2. Send Representatives to Professional Meetings to 
Recruit 

3. Send letters to “interested individuals identified 
through various personal contacts., 

4. Direct contact with Ph.D. programs 

5. Recruit ICAPP Advantage students (without 
threatening AACSB accreditation) 



6. Sponsor receptions at professional meetings. 

SUMMARY AND CONCLUSIONS 

Since being announced on April 29, 1999, much 
progress has already been made toward creating a School 
of Information Technology at Georgia Southern 
University. The school is an aspect of Governor Ray 
Barnes’ high technology economic development 
initiatives and is designed to address shortages of IT 
workers in Georgia as well as to serve as a magnet for 
attracting and expanding IT-producer andheavy IT-user 
organizations. This new academic unit is tasked with 
annually producing at least 300 (undergraduate) degrees 
by the 2004/2005 academic year. 

In response to the announcement, the Dean of the 
CollegeofBusiness Administration appointed an ad hoc 
committee charged with producing a white paper 
outlining a curriculum model for the new academic unit 
as well as degree program proposals. The degree 
programs recommended in the white paper blend 
business knowledge with information technology 
expertise to address the needs of the tai^eted industries. 
Initial input from external sources, including industry 
management and consultants, appear to validate these 
proposed curriculum model and degree programs. 

The white paper also includes student enrollment 
projections, projected faculty needs, an outlineof facility 
and resource requirements, a proposal to create an 
elnstitute, and a preliminary marketing and recruiting 
plan. The white paper’s key recommendations include: 

1 . The creation of a School of InformationTechnology 
that offers applied degrees in information 
technology. This new school should leverage 
current strengths and initially be a part of the 
existing University structure. Future growth may 
lead to the development of a College of Information 
Technology. 

2. Continued expansion of the BBA/IS degree and the 
creation of a Bachelor of Science in Information 
Technology degree program that includes a business 
minor. This degree will be an applied degree 
designed to prepare the graduate for a leadership 
role in the information technology industry. 

3. The establishment of an elnstitute to offer services 
and outreach programs to the region and state. 
Major components of the elnstitute would be the 
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eBusiness incubator program andthe certificate 
programs. 

4. Aggressive promotion and recruitment of School of 
IT students and faculty. 

Limitations 

The actual content of Georgia Southern’s white paper 
may be of limited value to educators and administrators 
at other universities who are faced with the 
challenge/opportunity of rapidly growing their IS/IT 
programs. However, its format and overall framework 
may serve as an model that other colleges and 
university’s can emulate. 

Georgia Southern’s white paper is a work in progress. 
The university committee has just been appointed and 
COBA’s white paper is currently being digested by the 
members of that committee. It will undoubtedly be 
modified by the representatives of other key consti- 
tuencies on the committee, especially those from 
computer science and the College of Science of 
Technology. In spite of this, on May 13, 2000 asign was 
erected on the vacant lot across the pedestrium from the 
COBA Building that reads: “Proposed site of the School 
of Informatio n Technology. An innovative economic 
development initiative of Governor Roy Barnes. 
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ABSTRACT 

This paper presents a Web-based environment (Web-course) designed for large groups distance learning programs 
in which students may benefit from the availability of large user and information bases, without losing the advantages 
of working in small groups. The system implements concepts of Social Translucence developed by Erickson and 
Kellog[2J. Co ntrolled visibility, mobility and management of student-produced information were built into the course 
organization and communication tools giving the student more alternatives and control over the process. Although 
the tools used in Web-course are all available elsewhere, their combination in one single environmentcould give the 
learner more flexibility and the instructor higher effectiveness in assisting the student and managing the course. 

The results show that collaboration and participation are positively related with visibility, mobility and performance, 
indicating that the addition of translucence and mobility resources to learning environment can improve the 
effectiveness of the process. Participant surveys also indicate satisfaction with the tools and results. 



INTRODUCTION 

Collaborative work can increase the effectiveness of 
distance learning. In most environments students have 
access to a common repository of information, to the 
instructorand support center, andinteractmore intensely 
with a small group of other students (their co hort) as part 
of the learning process. Traditionally growth of the 
student population is handled by forming more groups, 
students are assigned permanently to a group at the 



beginningof the course and information produced in one 
group usually is not shared with others. As a result, 
information and skills that may be useful for learning 
and student motivation are therefore underutilized. 

This paper presents a W eb^based environment (Web- 
course) designed for large groups distance learning 
programs in which students may benefit from the 
availability of large user and information bases, without 
losing the advantages of working in small groups. The 



298 



Proceedings of the 15^^ Annual Conference of the International Academy for Information Management 



BEST COPY AVAILABLE 



system implements concepts of Social Translucence 
developed by Erickson and Kellog [2] Controlled 
visibility, mobility and managem ent of student-produced 
information were built into the course organization and 
communication tools that give the student more 
alternatives and control over the process. Although the 
tools used in Web -course are all available elsewhere, 
their combination in one single environment could give 
the learner more flexibility and the instructor higher 
effectiveness in assisting the student and managing the 
course. 

CONCEPTUAL BACKGROUND 

Groupwork and collaboration, important resources for 
Distance Education that have been explored by authors 
like Harasin [3], Murray [4] and others, are also part of 
the tools implemented in Web-course. 

Another important concept for Web- course is “socially 
translucent systems . The concept “translucent is 
usually applied to human relations, and represents the 
amount of information about himself a person shares 
with others in a social environment. According to 
Ericksonand Kellog[2], social systemscanbeclassified 
depending on the level of members visibility, ranging 
from total opacity (no information available on|^ 
participants) to total transparency (participants are totally 
visible). Translucence could then be used as a 
mechanism regulating the interpersonal relationships in 
a social environment. The present paper explores the 
concept in relation to group work and collaboration in a 
Distance Education environment. Translucence was 
implemented in the form of controlled participant 
visibility, access to and organization of persistent 
conversation, aided by mobility among groups and 
information grouping. 

DEFINITIONS 

Visibility: The amount of information about 
himself a person is willing to share with others. 
Visibility ranges from total opacity (Fm not 
willing to disclose anything about myself) to 
total visibility (I want to expose myself 
completely to others). In a social environment, 
members are continually controlling their 
visibility according to contingencies. In a 
remote environment members are less likely to 
know each other which may condition their 
willingness to expose themselves. 



Persistent conversation: Messages exchanged 
among participants are recorded and organized 
to be accessed later. The recording may impact 
the participants communication behavior 
(Erickson and Herring [1]. 

Mobility: In the Web-course environment 
mobility represents the freedom participants 
have in choosing with whom to cooperate 
(switching groups or initiating new ones). For 
this choice the participants need to have 
information about the other participants and 
existing groups. 

Selective Information sharing: Groups can 
decide on the amount of information they make 
available to others. 

The experiment with Web-course used synchronous 
(chat) and asynchronous communication tools (forum, 
bulletin board, e-mail withdiscussion list and a common 
area for up- and downloading files). These tools allow 
various degrees of information structuring, ranging from 
chat to forum. The amount of structure depends on 
participants choice of communication protocol and the 
tools provided for communication. 

The environment was developed using the concepts of 
Visibility and Persistent Communication from Socially 
Translucent Systems, Computer Mediated 
Communication tools (chat, e-mail, discussion lists, 
fomm, bulletin board), allowing the presentation of 
instructional content, instructor support and performance 
evaluation. The environment allows participation, 
collaboration , controlled visibility and mobility. 
Dependent variables are participants satisfaction and 
learning performance. 

THE WEB-COURSE ENVIRONMENT 

Web-course is a Web-based learning environment for 
group or individual learning,, using software built on 
WindowsNT and implemented using Microsoft ASP, 
Java and SQL Server. 

Most communication and presentation tools were 
custom-developed for Web-course. The only exception 
was the e-mail baseddiscussion list, whichuses a Linux- 
based version of Majordomo. All user interfaces are 
based on standard Web browser. The experiment showed 
also that participants sometimes bypassed the system to 
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use commercially available chatsystems, thus avoiding 
the persistent communication features of Web-course. 
Participants can control fortranslucence using individual 
visibility control, persistent conversation and group 
mobility. 

Each participantcan control his/her own visibility to the 
group. Only name, login code and e-mail address are 
always visible to all others. This choice of visibility is 
made in two situations: 1. The participantcan supply a 
text to Web -course with the information about himself 
he wants to share with all others. 2. Through the choice 
of tools and content of the messages in the 
communication process. 

An important resource is the bulletin board, which is 
used for small and focused messages, like scheduling 
meetings, supplying hints on how-to find information or 
asking for help. Messages on the bulletin board are 
sorted by posting date, display ing author, group and text. 
Messages can be made visible to a single group or all 
participants. 

The forum is also persistent, allowing in addition the 
structuring of information through a topic hierarchy. 
Participants may ere ate any number of topics, with up to 
1 1 sub-topics and up to 999 messages in eachnode. Both 
bulletin board and forum can be used by participant to 
search for groups that best suit their specific needs. 

The chat is synchronous and all messages are 
permanently stored and can be retrieved later. Therefore 
the chat can also be used as an asynchronous 
communication tool. 

Persistent conversations allow participants to know the 
history of a given process and use messages generated 
synchronously in an asynchronous mode. 

IMPLEMENTING WORKGROUPS 
AND MOBILITY 

Web-course has also resources for groupwork: 
participants are allowed to create new groups anytime, 
and can switch groups as often as they wish. A group is 
identified by its name, creator id and a desc ription of its 
purpose. In our experiment the group creator had no 
control over its future content or participants decision to 
enter or leave the group. The experiment used two types 
of groups: those created by the participants themselves 
and those created by the instructor (all participants were 
originally assigned to one of them). The other was a 
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temporary group (INFO_WEBCOURSE) designed 
specially for synchronous meetings with the instructor 
(chat). Participants would enter this group for the 
meeting only and return immediately to their groups. 

In some instances a group became then simply a 
(temporary) work environment with its tools (forum, 
chat and bulletin board) storing also the groups history 
and its context. 

In our experiment participants were not allowed to 
participate simultan eously in more th an one gro up. This 
limitation (to be removed in future versions of the 
system) required the participants to switch groups in 
order to access other groups information. 

THE EXPERIMENT AND DATA COLECTION 

The basic hypothesis is that an environment with higher 
(selective) visibility and mobility among groups allows 
participants to increase collaboration, participation, 
learning performance and satisfaction. 

The experiment was used to test the following 
relationships among variables: 



1. Increased Visibility is related 
collaboration 


to 


increased 


2. Increased Visibility is related to increased mobility 


3. Increased Mobility is related 
collaboration 


to 


increased 


4. Increased Participation is related 
collaboration 


to 


increased 


5. Increased Mobility is related 
Participation 


to 


increased 


6. Increased Visibility is related 
Participation 


to 


increased 


7. Increased Learning Performance 
increased the variables above 


is 


related to 



The experiment consisted of a short (20 to 40 hours) 
skills-building course on “Conducting Effective 
Meetings , composed of five modules, the firstof which 
was a group dynamics for team building purposes and 
exploration of the computing environment. The others 
consisted of specific tasks, requiring th e reading o f texts 
(available for download), guided tours, searching Web 
sites provided by the instructor and a listof questions to 
be answered individually by the participants (these 
answers were graded by the instructor) . Since some of 
the questions were rather polemic, participants would 
benefit from discussing them with their peers. Each 
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participant was required to turn in his own answers, but 
interaction and collaboration was encouraged. 

The course can be characterized as semi-asynchronous 
with support, because of its eventual use of synchronous 
tools. The instructor offered assistance both by 
asynchronous answers to questions and previously 
scheduled synchronous chat sessions. 

Course participation was offered to teachers that had 
been assigned by the Ministry of Education to a 
Government-sponsored School Informatization Program 
[5]. These teachers were in charge of regional 
Educational IT support centers. They knew some oftheir 
colleagues from their State, and might have met others 
at the yearly National Educational Technology 
Conferences. 143 teachers (out of a total population of 
600), from all over the Country volunteered for the 
course. 75 of them concluding successfully the course 
and 73 of them answered the questionnaires. 

The Internet was used for participants registration and all 
other course communications. 

The total duration of the course was 23 days, with strict 
deadlines for each of the five modules. 

In order to avoid problems with anonymous 
participation, participants had to register formally and 
those who concluded successfully the course received 
certificates issued by the sponsoring University. These 
certificates could be used in future promotion processes. 
The data were collected in three stages: First stage: At 
registration time, the participants supplied demographic 
information and answered questions related to their 
expectations for an on-line Internet-based course. 
Second stage: The system recorded all accesses and 
participants communications. Third stage: At the end of 
the experiment, participants answered questions related 
to their participation and results of the course. In 
addition, participants who had initiated groups were 
interviewed for their views of this process. 

Table 1 contains the measures made during the 
experiment and the five dimensions in which these 
measures were grouped. 

The dimension “participation measures the students 
involvement with activities related to the completion of 



his own tasks. The dimension “collaboration measures 
the use of the systems tools for interaction or 
collaborati on with other participants. Mobility measures 
the frequency of group switches. V isibility measures the 
amount of personal information shared by the participant 
and his attitude toward sharing (measured by 
questionnaire). User satisfaction, attitudes toward 
collaboration, etc were measured in the questionnaires. 
These variables were combined forming the aggregate 
variables: participation, collaboration, mobility, visibility 
and performance. The other qualitative information was 
obtained through the pre- and after course 
questionnaires. 

TABLE 1 

MEASURES FOR MODEL VARIABLES 



Dimension 


Measure 


Participation 


Number of Accesses to Web-course 


Participation 


Number of Accesses to the Bulletin 
Board 


Participation 


Number of Accesses to the Forum 


Participation 


Number of accesses to chat sessions 


Participation 


Number of requests for support 


Participation 


Number of concluded tasks 


Collaboration 


Number of e-mail sentto discussion 
lists 


Collaboration 


Number of files contributed to 
forum 


Collaboration 


Number of sent chat messages 


Collaboration 


Number of questions or answers 
sent to bulletin board 


Mobility 


Number of group switches 


Visibility 


Personal information supplied to 
Web-course 


Performance 


Average participants grades 



RESULTS 



The objective of the experiment was to identify the level 
of satisfaction of participants with the environment, the 
relationship among the dimensions: participation, 
collaboration, mobility and visibility and the influenceof 
these factors on participants learning performance. 

Overall participants satisfaction with the course: on a 
scale from 0 (worst) to 10 (best), participants rated the 
course (Table 2). 
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TABLE 2 

PARTICIPANT COURSE RATINGS 



Rating 


Frequency (%) 


Cumulative 
Frequency (%) 


0 to 6 


0 


0 


7 


6.8 


6.8 


8 


27.4 


34.2 


9 


31.5 


65.8 


10 


34.2 


100 



This indicates a high satisfaction with the experiment. 
It should be noted that the participants considered the 
course subject important to theirjobs and valued 
highly the course content and structure and specially 
the instructor support, 



COMMUNICATION PATTERNS 
TABLE 3 

PARTICIPANTS NOT SENDING A SINGLE 
MESSAGE USING EACH TOOL 
(% OF TOTAL NUMBER OF PARTICIPANTS) 



Tool Participants Participants 

concluding abandoning 

course course 



Chat 


38.7 


72.1 


Bulletin Board 


54.7 


80.9 


Forum 


60 


82.4 


Support service 
(e-mail) 


65 


100 


Discussion list 


77.3 


100 



PARTICIPANTS DATA 

70,3% have a computer and Internet access at home. 
Only 21% had previous experience with distance 
education 

Only 8% had less than 2 years of experience with 
computers 

14% had less than 1 year of experience with the 
Internet 

80% accessed the Internet every day 

66% were connected to the Internet between 20 and 

60 hours per month 

70% worked between 20 and 40 hours to complete 
the course 

80% would access the Web-course site more than 5 
days a week 

55% declared having used other too Is, besides Web- 
course (ICQ, MIRC) for their coursework 
47,6% of the participants abandoned the course for 
various reasons. 

1 3 Participants created a total of 1 4 new groups, of 
which only two attracted a significant number of 
members. 8 of these group creators concluded the 
course . 



TABLE 4 

NUMBER OF MESSAGES SENT USING EACH 
TOOL 



Tool 


Average per 
participant 


Standard 

deviation 


Chat 


26.7 


64.4 


Bulletin Board 


2.1 


3.8 


Forum 


1.4 


2.2 


Support 


0.7 


1.4 


Discussion List 


0.6 


1.7 



Visibility: 58.7% of concluding participants did not 
disclose additional personal inform ation (77.9% for non- 
concluding). 80% of concluding participants didnot read 
other participants personal information and favor for 
collaboration those who had disclosed personal 
information. 

PARTICIPANTS MOBILITY 

Table 5 shows participants mobility among groups. At 
the beginning of the course all participants were assigned 
to group GOl . G02 was a special group, created and used 
by the instructor for his scheduled chats with 
participants. Participants had to switch to G02 in order 
to participate in each chat session. 
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TABLE 5 

MOBILITY AMONG GROUPS. (ROW = FROM, COLUMN = TO) 



From/ 


GO 


GO GO 


GO 


GO 


GO 


GO 


To 


1 


2 3 


4 


5 


6 


7 


GOl 




50 ! 1 


! 28 


10 


■ 3 ! 


1 3 


G02 


22 


|4 " 


29 


6 


A 


\2 


G03 


1 


5 1 


Tr 








G04 


: 13 


31 ! 


i 2 




2 


[l 


G05 


1 


' 6 i 


' 








G06 


' 2 


2 i 




2 




l' ’ 


G07 


; 2 


1 


‘ 4' 






\ 


G08 


2 


2 ' : 


! 


2 






G09 


3 


5 , 1 


\ 2 








GIO 


2 


2 ' 






1 ' 


\ 


Gil 




T 

1 








1 


G12 


2 


2 ’ ’ 


2 


2 






G13 




6 ; 








1 1 


G14 




1 ! 








t 


TOT 


56 


11 6 


68 


22 


10 


8 



8 

2 

2 

; i"" 

1 1 



10 



106 

83 

7 

53 

15 

9 

8 

7 
11 
6 

1 

8 
9 
2 



DEFINITION OF VARIABLES AND 
TESTING THE HYPOTHESES 

The concepts were measured by indices, constructed 
from the basic measures. These indices (one for each 
concept), were built as weighted sums of the variables 
for each dimension (Table l).The weights were defined 
by the authors as a measure oftheir relative importance. 
The average for each index was defined as the weighted 
sum of the original variables. This average was used to 
create a dicotom ic value (high or low) for each 
observation, to be used to test the hypotheses. Each 
combination would then result in a 2 X 2 matrix, to be 
tested using Qui-square (with correction for continuity 
for N>40), with the use of the SPSS v 8.0 package. 

For each participant there would be calculated a value 
(V) for the following indices. 

Ipart: degree of participation 
Icollab: degree of collaboration 
Imobi: degree of mobility 
Ivisi: degree of visibility 

For each index there was also obtained the weighted 
average (C), the cutting point, used to define the 
high/low values for each observation. In detail, 



the index and cutting point for Participation were 
calculated as: 

14,43*0,10 + 14,17*0,20 + 3,76*0,20 + 

4,34*0,20 + 0,36*0,20 + 2,85*0,10 

LPLOGON*0,10 + LPBULLET*0,20 + 

LPFORUM*0,20 + LPCHAT*0,20 + 

LPSUPPORT*0,20 + LPACTIVIT*0,10 

Ipar. = low, 

Ipan = high. >= (1) 

Where LPLOGON is the Number of Accesses to Web- 
course, LPBULLET is the Number of Accesses to the 
Bulletin Board, LPFORUM is the Number of Accesses 
to the Forum, LPCHAT is the Number of accesses to 
chat sessions, LPSUPPORT is the Number of requests 
for support and'LPACTIVIT is the Number ofconcluded 
tasks. The values used to define C are the averages of the 
variables observed in the sample. 

Icollab was defined in a similar way, with the variables 
presented in Table 1. 

Ccoiiab= 0,30 * 0,10 + 16,63 * 0,30 + 0,99 * 0,30+ 1,63 
* 0,30 
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BEST COPY AVAILABLE 



V„||3b= LCEMAIL ♦ 0,10 + LCCHAT ♦ 0,30 + 
LCFORUM ♦ 0,30 + LCBULLET ♦ 0,30 

Icoiub= low, if 

^collab high, if 

'^collab ^ ®collab (2) 

Were LCEMAIL is the Number of e-mail sent to 
discussion lists, LCCHAT is the Number of sent chat 
messages, LCFORUM is theNumber of files contributed 
to forum, LCBULLET is the Number of questions or 
answers sent to bulletin board. The values used to define 
C are the averages of the variables observed in the 
sample. 

According to Table 1, M obility is defined by one single 
variable: the number of group switches (LMGROUP), 
with an average value of 2. Therefore, 

Ubi = low, if LMGROUP < 2 

Ubi = high, if LMGROUP >= 2 (3) 

The visibility index was defined as follows: “high 
for each participant providing a text about him/herself 
and “low otherwise. 

The following relationships were found to be statistically 
significant (at the 1% significance level) in the sample. 

Visibility and collaboration 
Mobility and collaboration 
Participation and collaboration 
Mobility and participation 
Visibility and participation 

The relationship between mobility and visibility was not 
found to be statistically significant (at the 1% level). 
These results allow the inference that participants that 
expose themselves little, participate little, tendto be less 
collaborative and more static (low mobility among 
groups). Performance is also significantly correlated 
with participation and collaboration. 

QUALITATIVE RESULTS 

From the questionnaires and user interviews the authors 
could also conclude the following. 
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1 . The participants valued highly the support provided 
by the instructor and his team, particularly the fast 
return. During the 23 days of the course, the team 
provided about 100 hours of support, mainly by e- 
mail. This helped motivating the participant to 
remain active . 

2. Technology is still a problem : Participants indicated 
they had problems with their Internet connection, 
low bandwidth, system and application 
configuration, etc. which could be particularly 
annoying during synchronous chat sessions with 
peers or the instructor. 

3. Participants demonstrated that they prefer to work 
synchronously (using chat). Preferred times were 
after lunch (1 to 2pm) or early evening (8pm). Chat 
session with the instructor (held every 2 or 3 days) 
attracted between 2 and 15 participants and were 
highly valued. 

4. The participants preference for synchronous chat 
and little use of mobility may indicate that they 
value more traditional (hierarchical) teaching/ 
learning methods. 

5. Some participants had difficulty in structuring their 
communication and selecting the most appropriate 
tool (using instead the tools they knew best, like the 
bulletin board for chat-like communication, etc) 

6. Structuring the group activities was also a problem 
for most groups(defining protocols for the use of 
each tool, defining forum topics, scheduling chats, 
etc) This indicates that there is a learning curve for 
this type of tools, even for experienced computer 
users. In fact, participants use the tools they are 
more familiar with, instead of the most appropriate 
ones: e-mail was preferred over forum, even for 
longer “threaded ^ discussions, at the loss of 
structure and clarity. 

7. Participants tended to cluster in rather stable groups 
characterized by regional origin, in detriment of 
subject oriented groups. 

Tables 6 and 7 were obtained from the user 
questionnaires. The (high) motivation of participants 
during the course may be related to their perception' of 
course quality and perceived performance. Based on 
self-evaluation, 74% of the participants concluding the 
course rated their own performance with grades above 8 
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(on a 0 to 10 scale) and only 26% rated it between 6 and 
7. The course content also received high marks, with 
82% considering itexcellent (grades 9 and 10) and 1 8% 
rating it good (grades 7 and 8). 

TABLE 6 

SELF-EVALUATION OF PARTICIPANTS 
CONCLUDING THE COURSE 



Grade 


Frequency 


% 


% accum. 


6 


7 _ 


9,6- 


9,6 


7 


12 


16,4‘ 


26,0 


8 


42 


57,5’ 


83,6 


9 


8 


11,0 


94,5 


10 


’’’ 4 ' ’ 





100,0 


Total 


73 


100,0 





The results indicated also thatparticipants have to learn 
to work in this richer environment (creating and 
switching groups, searching for course-wide group 
generated information, publishing group information, 
etc.). Successful participant-created groups were usually 
based on existing social ties (regional groups) or had a 
very narrow task focus. M any groups did not succeed in 
attracting participation. Searching participant-produced 
information and switching groups was heavily used only 
by a small group of participants. 

TABLE 7 

EVALUATION OF COURSE CONTENT BY 
PARTICIPANTS CONCLUDING THE COURSE 



Grade 


Frequency 


% 


% accum 


7 


3 


4,1 


4,1 


8 


10 


13,7 


17,8 


9 


21 


28,8 


46^6 


10 

Total 


39 

"'_'""73 ' 


53,4 

100,0- 


100,0 



CONCLUSIONS 

The results show that collaboration and participation are 
positively related with visibility, mobility and 
performance, indicating that the addition of translucence 
and mobility resources to learning environment can 
improve the effectiveness of the process. The results 



indicate that users are satisfied with the Web-course 
environment, allowing them more flexibility in 
workgroup collaboration and access to user-generated 
information. The multiple tools give the instructor a 
better view of students performance and allows more 
effective intervention in the learning process. The 
usefulness of the different tools will depend on the 
characteristics of the course activities, on the students 
skills in using them and on students learning behavior, 
since they must learn to take advantage of the new 
possibilities of collaborative work, breaking away from 
imposed static structures. Course designers will also 
have new possibilities for designing students activities. 
Students prefer workgroups thathave social cohesion or 
a strong task focus. Future developments of Web -course 
will allow more participants control over the group 
forming and switching process and better tools for 
(selective) sharing of information produced by the 
groups and also access to statistical data on group 
behavior. 
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ABSTRACT 

Distributed collaboration supported by different forms of information and communication technologies (ICT) is 
becoming increasingly widespread. Effective realization of technology supported, distributed collaboration 
requires learning and careful attention to both techno logical and organizational aspects of the collaboration. 
Despite increasing focus on the use of ICT in education, for example in the form of distance learning and 
computer supported collaborative learning, there still are few educational programs that includean explicit focus 
on providing the students with practical training in distributed collaboration. This article presents the results and 
experiences from two student projects conducted at Agder University College in Norway, where groups of 
students from campuses in two different cities engaged in distributed collaboration supported by different forms 
of collaboration technology. The project illustrates how students perceived this form of distributed work to be 
interesting and stimulating, but also challenging and time consuming. Task definitions and balancing of 
motivational factors were found to have a great impact on the groups* efforts, sharing of workload and final 
outcome. Further, the projects also illustrate how **technical noise** and lack of experience with the technologies 
represented barriers to effective collaboration, implying a need for training the students in integrated use of 
different communication services. Based on the experiences from the two pilot projects, the paper presents 
practical implications for developing and conducting similar projects. 



INTRODUCTION 

Different forms of IT supported collaboration across 
geographical distance become increasingly widespread 
in today's organizations. This can be seen as a result of 
several factors. The development towards global and 
dynamic markets creates a need for flexibility in time 
and place for organizational work tasks. Further, the 
technological development has now reached a stage 
where different communication tools can be acquired 
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"off the shelf, and where bandwidth and quality of 
communication no longer represent unobtrusive 
barriers to distributed interaction. Through combined 
use of different collaboration technologies like e-mail, 
desktop videoconferencing systems, application 
sharing and document exchange, distributed 
collaboration today can be equally effective as co- 
located work. Distributed collaboration here comprises 
different forms as related to organizational scope and 
time frame, and may include both intraorganizational 
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collaboration between geographically dispersed units 
in large organizations, and project based networks 
among small and medium sized enterprises (Line, 
1999a; Munkvold, 1999). 

In academic research, virtual teamwork has long been 
presented as a key element in the "organizations ofthe 
future" (e.g., Cohen, 1993; Davidow & Malone, 1992; 
Keen 1991; Vitalari, 1990). However, it is only during 
the last five years or so that this has become reality, as 
a result of the technological development described 
above. These solutions thus have developed from 
"hype" into being characterized as "mission critical" 
by many organizations today. It is important here to 
realize that effective deployment of distributed 
collaboration does not come about by itself, but 
requires active learning and understanding of both 
technological and social aspects by the actors 
involved. It is therefore necessary for educational 
institutions to develop study programmes that provide 
the students with adequate background and training in 
this new form of work. While there is much activity 
today centred around the use of ICT in distance 
learning as a substitute for traditional classroom 
teaching, there are still few study programmes that 
offer explicit and practical training in effective, 
distributed collaboration. 

This paper presents the experiences from two pilot 
projects in distributed collaboration among students, 
conducted at Agder University College (AUC) in 
Norway during fall 1998 and fall 1999. In these 
projects, groups comprising students from two 
different courses given at AUC's campuses in the cities 
of Kristiansand and Grimstad worked together on a 
project assignment over a period of two-three weeks. 
During this period, the groups collaborated entirely 
over the net, supported by a selection of different 
technologies for distributed collaboration: e-mail, 
application sharing, audio and videoconferencing, 
document repositories and discussion databases. 
Through presentation and discussion of the 
experiences from these projects, the goal is here to 
establish a foundation for further development of this 
type of educational oflFerings, and to stimulate 
discussion on the conduct of similar projects. 

The article is structured as follows. The next section 
provides a brief overview of related research on 
distributed collaboration and use of ICT in education. 
This is followed by a description of the two pilot 
projects, and a subsequent discussion of the 



experiences from the conduct of these projects. The 
last section presents implications for developing 
similar projects for training students in distributed 
collaboration, as well as implications for further 
research and practice. 

OVERVIEW OF RELATED RESEARCH 

Distributed collaboration supported by IT is at focus in 
several research arenas. This section presents a brief 
overview of some of these research activities. 

Computer-Mediated Communication 

In the area of Computer-Mediated Communication 
(CMC), studies of how "electronic communication" 
affects interpersonal interaction and collaboration in 
distributed settings have been conducted for more than 
two decades (e.g., Hiltz & Turoff, 1978; Short et al., 
1976). Most of this research is of an experimental 
nature, being conducted in controlled environments 
(e.g., classroom settings) with students as subjects, 
working on artificial tasks. A large part of the research 
on distributed collaboration has focused on some form 
of teamwork, and different variables related to group 
dynamics and group characteristics have therefore 
constituted key elements in the research models 
developed. A range of different factors have been 
studied in these experiments, such as the effects of 
technology support on effectiveness and quality in 
decision processes, and participants' perception of 
process and outcome. A meta-analysis of the results 
from a selection of these studies indicates that users 
normally are capable of adapting to the changes in 
communication form implied by use of technology 
support, and that use of electronic communication 
does not result in reduced quality of outcome. In some 
cases it may even lead to better quality through 
stronger task focus and the possibility for more in- 
depth analysis (Munkvold, 1996). On the other hand, 
there are also several examples of how the time used 
for making decisions in this form of collaboration 
increases, and of team members reporting reduced 
satisfaction with the group process. 

The experimental context in which these results are 
generated clearly limits the transferability to 
distributed teamwork in real organizations. Typical for 
these studies is also a focus on comparison between 
distributed and co-located ("face-to-face") teamwork. 
In several organizations, distributed collaboration will 
represent the only possible alternative due to 
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geographical barriers (Line, 1999a). Further, it is 
worth noting that the technology used in these studies 
often is limited to e-mail and text based conferencing 
systems, and does not include richer communication 
media like desktop videoconferencing and application 
sharing . 

Virtual Teamwork 

During the last five years there has been a growing 
interest in so-called virtual teamwork (e.g., Lipnack & 
Stamps, 1997), where team members collaborate from 
several geographical locations supported by different 
types of communication technologies. Much of the 
research related to this form of collaboration still 
suffers from the same limitations as earlier CMC- 
studies, i.e. lack of realistic context and limitations in 
technology support. Several projects that are being 
presented as ’’advanced" examples of virtual 
collaboration are actually based on e-mail as the only 
form of technology support (e.g.. Knoll & Jarvenpaa, 
1998). 

Concurrent Engineering 

While virtual collaboration in administrative office 
work until recently has had a rather ’’exotic" character, 
there is a long tradition in the engineering disciplines 
for this form of work. For example, in Norway the 
construction and oil industries have been early 
adopters of ICT for supporting coordination and 
information exchange between different actors. In 
general, the area termed Concurrent Engineering (CE) 
comprises several examples from different industries 
of relatively advanced, distributed collaboration 
projects regarding technology support and complexity 

(e.g., Ashby et al., 1995). 

/ 

ICT in Education 

In education today there is an increasing focus on the 
use of ICT both to enable new teaching methods and 
to increase efficiency for traditional instructional 
methods. This relates both to distance learning (e.g., 
Dede, 1996) and computer supported collaborative 
learning (CSCL) (e.g., Brandon & Hollingshead, 1999; 
McConnell, 1994; Seufert & Seufert, 1998). Fjuk 
(1998) introduces the term Computer Support for 
distributed Collaborative Learning (CSdCL) for 
denoting the intersection between these two areas, thus 
also including physical distance among the 
collaborating students. There is also an increasing use 



of Internet and the World Wide Web for supporting 
various forms of learning approaches (Bandi & 
Nakatani, 1998; Dufner et. al., 1999; O’Leary & 
Fischer, 1998). In general, the activity related to the 
use of ICT in educational settings is largely focused on 
supporting the learning process (e.g., Beranek & Loch, 
1999; Vreede et al., 1999). There are still few courses 
that focus on how the technology can be used 
effectively to support distributed collaboration. 

Conclusion 

A conclusion from this brief overview of related 
research is thus that there exist few course offerings 
that focus explicitly on distributed collaboration in 
practice. Further, we argue that the research related to 
distributed collaboration supported by ICT is still at an 
early stage, at least if realism in context and use of 
state of the art technologies are applied as criteria for 
maturity. However, we need to stress that the projects 
reported in this study have had the character of 
practical pilot projects in an educational context, and 
therefore have not followed astringent research design 
regarding data collection and analysis. The primary 
purpose of the projects has thus been to generate 
experiences that can form a basis for further 
development of instructional methods for training 
students in practical, distributed collaboration. 

STUDENTS IN THE WILD — DESCRIPTION OF 
THE TWO PILOT PROJECTS 

This section provides a description of the scope and 
conduct of the two student projects in distributed 
collaboration conducted at AUC during the fall terms 
of 1998 and 1999. Discussion of experiences and 
results from the projects are presented in the next 
section. 

Background and Purpose 

The idea for these projects was conceived during a 
discussion on how to develop common teaching 
modules for the following two courses: 

Coordination Technology - second year course in 
the two year Master of Engineering programme in 
Information and Communication Technology 
(ICT) at the AUC campus in Grimstad 

Computer-Supported Cooperative Work (CSCW) - 
first year course in the two year Masters 
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programme in Information Systems (IS) at the 
AUC campus in Kristiansand 

By establishing distributed project groups across the 
two courses, the purpose was to give the students 
practical experience with distributed teamwork, 
including coordination and use of different forms of 
collaboration technology. The contents of the project 
assignment should be a relevant course topic and 
mimic a real world task as closely as possible. A third 
goal was to establish an arena for interaction between 
the students from these two programmes. This contact 
is normally restricted by the geographical distance (60 
km) between the two study locations. So far, this 
project has been conducted for two successive years, 
with adjustments being made based on the experiences 
from the first year. In the following we provide a brief 
description of the two projects, including an overall 
evaluation of the results. 

Pilot Project 1 

The first project was planned during summer 1998 and 
started approximately one month after start-up of the 
semester. The assignment given was to compare the 
functionality of two "groupware platforms": Microsoft 
Exchange/Outlook and Lotus Notes/Domino. These 
two technology platforms were used this year as the 
basic infrastructure in the two courses, in Grimstad 
and Kristiansand respectively. Table 1 presents the key 
contents of the project. 

Data Collection 

Different forms of data collection were conducted 
during the project period, as a basis for the evaluation 
of the pilot project. The experience reports were 
specified at the outset as to include the following 
topics: the group's organization of the project, use of 
technology support, experiences from the 
collaboration in the group, and reflections on how the 
group would conduct a similar project in the future. By 
having the participants from each course hand in 
separate experience reports, we wanted to give them 
the opportunity for presenting their experiences and 
views related to sharing of workload and collaboration 
in general with the participants from the other course. 
The students were guaranteed that these reports would 
be confidential, read by the two instructors only. 

In addition to the experience reports from the groups, a 
web based form for self-reporting of activities during 



the project period was created. The groups could also 
print out the form and fill this in manually. The groups 
were requested to hand in this activity log as an 
attachment to the experience report. Only three of the 
five groups used the form. This can be ascribed to 
several factors. First, the form was not available until a 
week after the proje ct had starte d. Second, the 
instructions to the students were not sufficiently 
explicit on the need for appointing a group member as 
responsible for the logging of activities. Consequently, 
for some groups this was given low priority compared 
to the work on the assignment itself. 

Overall Evaluation of the Project 

Evaluation of the groups' results in the form of project 
reports and presentations shows that the groups to a 
varying extent managed to solve the task assigned (the 
project reports are available at http://siving. 
hia.no/ikt97/it4200/). All the groups had chosen to 
divide the project between their distributed units, so 
that the participants from each course were responsible 
for the groupware product available at their site. This 
could be expected, but both the reports and the related 
presentations revealed that several of the groups had 
not succeeded in coordinating the contents and 
presentation regarding comparison of the two 
products. For several of the groups their result was 
limited to including a separate description of the 
functionality for each product. 

The project was reported by the students to be time- 
consuming and almost of an "absorbing" nature, at the 
expense of other course work. This must also be 
related to the fairly broad task assigned, requiring 
delimitation of focus by the students. According to 
several of the groups, the difference in evaluation form 
between the two courses also resulted in that 
motivation and effort put into the project was unequal 
among the participants from each course. 

Lecturing continued as normal during the project 
period and the students were responsible for 
coordination and finding time for the project work. 
Some groups experienced difficulties with finding 
available time for synchronous sessions/meetings, due 
to the different schedules for the two study 
programmes. 

Supervision was conducted in the form of the instructor 
at each site answering questions from students in his 
course. Thus, there was no supervision with the entire 
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group present. A problem with this form of ’’local” two instructors also occurred some times, thus leading to 

supervision was that different interpretations among the confusion and frustration among the students. 

TABLE 1 

KEY CONTENTS OF PILOT PROJECT 1 



Pilot project 1 

Assignment 

Comparison of Microsoft Exchange/Outlook and Lotus Notes/Domino on the following areas 
of functionality: E-mail, Internet integration, Application Development Environment, Database 
structure. Security 



Deliveries 

Written report, half hour presentation in class. Two experience reports per group, from the 
participants from each course. 



Evaluation/Incentives 

Part of the mandatory assignments for both courses. For ICT students the projects were graded, 
comprising 1 0 % of the total grade. For IS students the projects were graded as ’passed/fail’. 



Schedu le 

Prior to the project, students in both courses were given an introductory lecture on application 
sharing and some practical exercise with M icrosoft NetMeeting, The project activities were 
scheduled as follows: 

Kick-off seminar atAUC Kristiansand. Introductory lecture on distributed collaboration. 
Forming groups, selection of topics. Lunch in groups, introduction/getting acquainted, 2 hours 
with group work on problem specification and planning. Concluding guest lecture from Lotus 
Notes vendor. 

Distributed project work for three weeks. No face-to-face contact between participants from the 
two courses. Group members from same course allowed to work face-to-fece. Face-to-face 
supervision available from the course instructors in Grimstad and Kristiansand on request. 
Closing seminar at AUC Grimstad. Groups working for 2 hours finalizing their reports. Group 
presentations in class. Guest lecture from vendor focusing on comparison between MS 
Exchange/Outlook and Lotus Notes. 



Group composition 

6-7 person groups with 3-4 students from each course. 5 groups in all, with a total of 18 IS- 
students and 16 ICT students. 



Infrastructure 

The students in both courses had a dedicated PC lab at their disposal, offering access to the 
following technologies/applications: 

E-mail (MS Exchange Outlook in Grimstad and EudoraPro 3.0 in Kristiansand) 

Microsoft NetMeeting v. 2.01 
Lotus Notes v. 4.5 (Domino) 

Worldwide Web (WWW) 

Since MS Exchange/Outlook and Lotus Notes only were installed at one campus each, remote 
access to the tools was provided through use of MS NetMeeting and by accessing the products 
over the Internet through a web browser. To enable the last approach, guest accounts for remote 
login were created at each site. 
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Pilot Project 2 

The project was repeated during the next conduct of 
the two courses the following year. Table 2 outlines 
the key contents of the second pilot project. Several 
changes were made based on the experiences from the 
first pilot project. In addition, the group composition 
was also different, due to uneven numbers of students 
in the two courses this year. The changes made are 
summarized in the following: 

Task made simpler and more cohesive 
The assignment in pilot project 2 was chosen on 
the basis that it was simpler, requiring less work 
for clarification and delimitation offocus and thus 
making it easier for the students to start on the 
actual work. Further, the task should be "cohesive” 
for the group, and not of a nature that invited for 
dividing subtasks between participants from the 
two courses. 

Incentives made more equal 

Incentives were made more equal for the two 
student groups, by grading the project in the IS 
course as well. However, due to practical 
concerns, the assignments still comprised different 
percentages of the total grade. 

Shorter project duration 

The duration of the project was reduced to two 
weeks, to avoid taking up too much time during 
the semester. 

More timefor group work during the kick-off seminar 
The time for group work during the seminar was 
extended somewhat, to give more time for 
planning the project and building relationships. 

Coordination of course schedules 
The schedules for the two courses were 
coordinated to make it easy for the students to find 
time for synchronous collaboration. 

Group-wise supervision 

Each group was assigned one instructor as 
supervisor, to ensure consistent advice and avoid 
redundant information. 

Group composition 

Due to changes in the Master programme in 
Kristiansand, the number of IS students 
participating in the project this year was 



considerably less than for the ICT students, 
leading to uneven numbers in the groups. This 
made it necessary to form two groups comprising 
ICT students only. Most of the students in these 
groups worked from the same location (three of 
them were part time students and therefore worked 
in a "real" distributed setting), but were asked to 
"simulate" distributed collaboration by using 
electronic communication only. 

Appointment of "observers" 

One ICT student in each group was assigned the 
role as "observer" with responsibility for logging 
technology use and process data for the experience 
reports. 

Regarding infrastructure, the same technologies were 
available as for project 1 (with some upgrades) with 
the addition of two PCs with Intel Proshare, 
supporting desktop videoconferencing. Further, mobile 
telephones were found to be frequently used by the 
students, although not being part of the "designed" 
project infrastructure. 

Data Collection 

Similar to the first pilot project, the experience reports 
including the log of communication sessions were the 
main source of data. In addition, pre and post surveys 
were conducted for the second project. The pre survey 
was conducted at the end of the kick-off seminar, 
asking the students about their previous experience 
with distributed collaboration, and their expectations 
for the distributed project regarding efficiency, quality 
of outcome and personal satisfaction as compared to 
traditional, co-located teamwork. The post survey was 
conducted after completion of the project by using the 
GroupSystems survey tool. The IS students completed 
the survey in class, giving 8 out of 8. possible 
responses. Due to limited installation of GroupSystems 
in Grimstad, the GroupSystem s form had to be 
distributed by e-mail to the ICT students. 15 out of 31 
possible responses were returned, giving a total 
response rate for the survey of 59 %. The post survey 
covered the students' experiences from the project 
regarding perceived quality and efficiency, and their 
satisfaction with the process and learning outcome. 
The analysis showed no significant differences in the 
answers between the two student groups. The results 
from the two surveys will here only be used for 
supporting the analysis of the experience reports in the 
next section. 
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TABLE 2 

KEY CONTENTS OF PILOT PROJECT 2 



Pilot project 2 

Assignment 

Developing a web-site presenting information on one of the following technologies: 

Palm PCs, PC-based real-time conferencing, Electronic meeting support systems, Workflow, 
Document administration, Streaming multimedia. 

The web-site should cover the following parts: introduction and overview of the area, 
presentation of a framework for comparison of related products, product overviews and links 
to related research and communities. 



Deliveries 

Web-site, 20 minute presentation in class. Two experience reports from each group, from 
participants from each course. 



Evaluation/Incentives 

Part of the mandatory assignments for both courses. The project was graded in both courses, 
comprising 10 % of the total grade for ICT students and 20 % for the IS students. 



Schedu le 

Prior to the project, students in both courses were given an introductory lecture on application 
sharing. In addition, the IS students were given some practical exercise with Microsoft 
NetMeeting. The project activities were scheduled as follows: 

Kick-off seminar at AUC Kristiansand. Introductory lecture on distributed collaboration. 
Forming groups, selection of topics. Lunch in groups, introduction/getting acquainted. 2 1/2 
hours with group work on problem specification and planning. Concluding lecture on 
distributed teamwork. 

Distributed project work for two weeks. No face-to-face contact between participants from 
the two courses. Group members from same course allowed to work face-to-face. Each group 
assigned one instructor as supervisor. 

Closing seminar at AUC Grimstad. Video transferred guest lecture on digital communication 
and organizational challenges. Groups finalizing their presentations. Lunch. Group 
presentations in class. 



Group composition 

3 groups with 2 IS students and 4 ICT students, I group with 2 IS students and 3 ICT 
students, 2 groups with 2x4 ICT students in each. One ICT student as observer in each group. 
A total of 8 IS students and 31 ICT students. 



Infrastructure 

The students in both courses had a dedicated PC lab at their disposal, offering access to the 
following technologies/applications: 

E-mail (MS Exchange Outlook in Grimstad and Eudora Pro 3.0 in Kristiansand) 

Microsoft NetMeeting v. 2.1 1/3.01 

2 Intel Proshare workstations placed in dedicated rooms close to the PC labs at each site 
Lotus Notes v. 4.5 (Domino) 

Worldwide Web (WWW) 

Mobile phones (used by the students as a supplement to the specified infrastructure) 
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Overall Evaluation of the Project 

All groups presented a common web-site for their 
work and in general all main questions were answered 
(web-sites (in Norwegian) are available at http://ikt98. 
grm.hia.no/ikt4200/). Most of the presentations were 
of good quality and reflected that considerable work 
had been put into the projects. However, the quality of 
the web-sites varied significantly in contents and 
structure. The presentations showed that most of the 
groups had divided the tasks between the students in 
the two courses, so that the ICT students were 
responsible for the technical part related to the design 
and development of the web-pages, with the IS 
students as content providers. Still, the end product 
was more integrated and coherent than for the first 
pilot project. 

Despite the course schedules being coordinated at the 
outset, some groups still reported difficulties in finding 
available time for synchronous meetings. As the 
number of part time students was greater this year, 
especially among the IS students, this meant that 
several of the students were only able to work on the 
project after normal hours. 

One of the two groups comprising ICT students only 
reported that they had followed the instructions and 
simulated distributed collaboration, using electronic 
communication only. The other group felt it to be 
inconvenient and unnatural to use these 
communication services, as they in fact were working 
in the same PC lab. They had therefore chosen to 



complete the assignment in their "normal” way of 
working, using face-to-face meetings. 

In general, the groups had very few questions for the 
supervisors this time. This can be explained by the 
relatively simple nature of the task, at least as 
compared to the assignment in pilot project 1 . 

DISCUSSION OF EXPERIENCES 
FROM THE PROJECTS 

Use of Technology 

The experience reports draw a fairly unambiguous 
picture of how the groups used the communication 
services and their evaluation of the different services. 
An overview of the services and their characteristic 
use is given in Table 3. The "noise" column indicates 
the level of experienced technical problems disturbing 
the communication. These include both "real" 
technical problems and problems caused by 
insufficient user competence. Use patterns for the 
individual services are elaborated in the rest of this 
section. 

E-mail 

The participating students can be characterized as 
experienced e-mail users. All groups reported frequent 
use and point to e-mail as the basic and most important 
service. Although mainly used for coordination and 
short messages, some of the logged messages also 
contain examples of "rich communication" (Lee, 



TABLE 3 

OVERVIEW OF TECHNOLOGY USE 



Service 


“Noise” 


General use 


Pilot project 1 


Pilot project 2 


E-mail 


No 


Service most used; messages, coordination, exchange of documents and information, short 
discussions 


NetMeeting 


Considerable 


Importantservice for most groups; 
application sharing, audio and 
some chat 


Mostly used for one to one 
meetings; explaining 
concepts and demonstrating 
product 


Mostly used as support for 
meetings with several 
participants 


Video- 

conference 


Considerable 


Group meetings, not important 


N/A 


Experimental 


Document 

repositories 


Some 


Repositories for common documents. Of moderate to high importance to somegroups, not used 
by others. 


Web 


Little 


Information search, publishing of 
results, web interface to other 
services 


Important, but not the only 
source of information 


The only source of 
information 


Mobile 

phones 


No 


Messages, coordination, "get in 
contact, backup audio channel 


No use reported 


Use reported by all groups 
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1994). For example, we noticed a "heated discussion" 
that probably should not have been conducted through 
e-mail (the supervisor was cc’ed to be informed about 
lack of commitment from two members of a group). 
Another group reported a "dangerous situation" where 
the air was cleared by a phone call. Several groups 
pointed to or discussed limitations in the use of e-mail, 
e.g., difficulty in conducting longer discussions, a 
contextual information and information overload. 
However, it is difficult here to separate real experience 
from "show off" of the course curriculum. 

NetMeeting (NM) 

The students had relatively little experience in use of 
NM. Hence most of the groups used some time 
"troubling" and experimenting with the technology. 
NM supports multipart application sharing, whiteboard 
and chat. Unless you are connected to a Multipart 
Conferencing Unit (MCU), the audio and video 
connections only support two participants. This 
information was given to the students in lectures prior 
to both pilot projects. Still, most of the groups in the 
second project reported that they were not aware of 
this limitation and that they had spent much time 
trying to get a multipart audio conference working. In 
general, the audio quality is reported as a source of 
"technical noise. Several case studies report that 
normal practice in industry is to support multipart NM 
conferences with load speaking phones (Line, 1998; 
Mark et al. 1999). The students did not have access to 
this type of equipment and did not have their own 
cubicles or office spaces. This difference limits the 
potential use of the service. 

Except for two groups (one in each pilot project), all 
groups perceived NM to be of practical use, and saw 
the potential for this type of service. The reported 
frequency of use was in average two-three times per 
week, with duration varying from 15 minutes to 3 
hours. 

In the first project, the most common use of NM was a 
meeting with two participants who both used headset 
with microphone. The typical use was mutual 
demonstrations of features in the two products that 
should be compared, discussion of the project report 
and discussion of the presentation material (Power- 
Point documents). These sessions were reported to be 
focused and efficient. In some cases this practice was 
reported to be a source of "free riding, as expressed 
by the following statement from one of the groups: 



the group members that do not participate in 
the discussions are not involved in the tasks in 
the same way. This may lead to less 
commitment and involvement. 

In the second project, NM was often used as a support 
for scheduled meetings that involved all group 
members. Overlooking the earlier mentioned 
"technical noise, the groups were mainly satisfied 
with these meetings. However, several noted that they 
regarded it as a formal tool, i.e. the communication 
channel did not sustain "small talk." 

Videoconferencing 

Videoconferencing was only available in the second 
conduct of the project. Some students had experience 
with use of the equipment, but unfortunately there was 
not time for systematic training of all groups. All 
groups used the equipment at least once. Some groups 
experienced "technical noise, probably caused by 
lack of experience and competence, "itchy fingers" and 
curiosity about the different features. The general 
impression is that they found it interesting and more 
informal then NM. The added value to the meetings 
was found to be relatively marginal. 

Document Repositories/Discussion Databases 

In both projects, several groups established some form 
of discussion database or document repository. These 
services were created by Lotus Notes/Domino/ 
Teamroom, Microsoft FrontPage and a Web based 
Extranet product. None of the groups reported any use 
of these services for real discussions. The normal use 
of these services was as a common document 
repository. To some of the groups, the work with 
establishing this service was also relevant for the 
assigned task (Internet integration and database 
structure in the first project, and document 
management in the second project). 

Worldwide Web (WWW) 

The important use of WWW was information retrieval 
and publishing of the deliveries. Although not 
explicitly reported, the information search capabilities 
were more important in the second project due to the 
nature of the assignment. Some groups mentioned 
problems with finding relevant, unbiased and qualified 
information. The referenced material is to a large 
extent clearly biased (vendor data sheets, white papers, 
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etc.). Though not investigated in detail, it is our 
impression that most of the groups were aware of this 
and treated this type of information with necessary 
caution. 

Almost all groups in the second pilot project appointed 
a ”webm aster” responsible for the final publishing of 
the deliveries, or delegated this task to the ICT 
students in the group. Given the ICT students’ former 
experience with and access to the web publishing tool, 
this can be seen as a natural decision and is not 
reported to have caused major problems for the work. 
Some groups used web interfaced document 
repositories. In the first pilot project, the web interface 
in Exchange/Outlook and Notes/Domino was used for 
remote access in comparison of the two products. Due 
to the limited features available in the web interface, 
most groups reported that NetMeeting was a more 
suited service to support this task. 

Mobile Phones 

The use of mobile phones is an interesting difference 
between the first and second conduct of the pilot 
project. In the second project, almost all groups 
reported use of mobile phones and that this use was 
efficient and valuable in various situations. All 
students were well acquainted with this ’’service” and 
therefore did not ’’stretch its limits” or "experiment” 
with its capabilities. The use was straight forward, 
such as contacting the other group, short messages, 
simple discussions and coordination of activities. 
Another valuable use was to serve as a backup 
communication channel in situations where the audio 
contact in NetMeeting or the videoconferencing 
system broke down. 

General Reflections on the Use of Services 

E-mail, and in the second pilot project also mobile 
phones, can be characterized as simple services with 
practically no technical noise. Given proper 
configuration and management of the network and 
workstations and sufficient training and experience in 
use of the services, it should be possible to remove the 
noise from the other services as well. This would 
probably have a significant impact on the combined 
use of the communication services. Earlier field 
studies show that it can take considerable time to 
develop new use patterns and exploit the full potential 
of a new service (e.g., Grudin & Palen, 1995; Line, 
1998, Munkvold, 1999). 



Evaluation of the Process 

In the following, the work process during the projects 
and the final results presented by the students are 
evaluated. This evaluation is based on the experience 
reports, the post survey in the second conduct and 
subsequent discussions with the students. Unless 
explicitly mentioned, the evaluation is based on both 
pilot projects. 

Organization of the Projects 

The groups chose a flat structure with no formal 
leader. Most of the groups appointed a contact person 
responsible for coordinating the activities at each site. 
Coordination and finding time for synchronous 
sessions were reported as issues in both conducts of 
the project. In the second project, most of the groups 
acknowledged this responsibility and made clear 
appointments. Meetings including all group members 
were more used in the second conduct. Few groups 
appointed a meeting leader, but several reported a need 
for this. 

In the second conduct most of the groups started with 
a division into subtasks and a relatively strong 
assignment of tasks. This assignment was mostly 
based on an evaluation of fair division of workload. 
Most groups appointed a web master. 

Conflicts 

The overall picture is that the conflict level was very 
low. Consensus was normally reached by e-mail 
discussions, NetMeetings or by phone calls. The main 
sources for the few escalated and surfaced conflicts we 
observed were: 

Broken appointments (in many cases due to 
unclear agreements) 

Lack of responsibility and commitment to 
assigned tasks 
"Us - them” issues 

The conflict level in the two co-located groups 
comprising students from only one class was higher 
than in the distributed groups. Although we cannot 
draw any firm conclusions, the material from the two 
projects shows no indications of a raised conflict level 
or difficulties in solving potential conflicts in a 
distributed work setting. 
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We expected that the different professional 
perspectives in the two study programmes could be a 
potential source of conflict. Several differences, 
misunderstandings and potential conflicts were 
reported. However, these comments are outweighed by 
positive statements of how the students found it 
interesting and instructive to meet and work with 
people taking on a different approach to the 
assignment. 

Efficiency 

The general feedback is that the efficiency of the work 
was experienced as acceptable, but slightly reduced by 
the distributed setting. When measured against the 
students’ expectations as expressed in the pre survey, 
the post survey indicates that the students experienced 
slightly higher efficiency impact than expected. The 
main cause of efficiency loss was technical noise and 
lack of experience with this way of working. The 
initial meeting at the kick-off seminar was also 
reported by most groups to be important for the 
communication in the distributed projectperiod. 

Quality ofOutcome 

In the first conduct of the project most of the groups 
were not content with the quality of the final product. 
The open ended and complex assignment together with 
technical noise with the communication services were 
stated as reasons for this. This is also in line with our 
evaluation of the project reports. Only two groups 
managed to compare the products using a common 
terminology. The rest of the groups mostly described 
"their" product using the vendor’s terms. 

For the second pilot project the assignment was made 
simpler and more specified. Jn this conduct the 
majority of the students were content with the quality 
of the group's work. Approximately 50% of the 
students thought the group’s result would have been 
better if the group had been co-located during the 
project. None of them refer to specific problems that 
affected the quality. The general response is that 
communication would have been easier in a co-located 
context. Our evaluation of these reports is that all 
groups have answered all questions and managed to 
deliver a common product. We noted that several of 
the groups had closed important questions early, such 
as selection of products to compare and framework for 
comparison, without consulting the supervisors first. 



Satisfaction with Learning Experience 

The general picture is positive. The projects were 
characterized as interesting and instructive and the 
students claim to have achieved a better understanding 
of possibilities and limitations with distributed 
collaboration. In the first conduct, the students 
reported that the project was very demanding, both 
with respect to the complexity of the assignment and 
the total time spent on the project. 

Technical noise was reported to be a source of 
frustration. On the other hand, we also have statements 
where experimenting with the communication services 
was regarded as an integrated objective in the project. 

As can be expected, the ICT students participating in 
co-located groups reported less positive experiences 
than the rest. The “faked distributed setting felt 
unnatural, and they did not get the right experience of 
being dependent on the communication services. 

Half of the students reported that they would have 
preferred to travel for working face-to-face, if given 
the resources. On the other hand, the survey shows that 
the students found working in a distributed team to be 
satisfactory. In other words, distributed collaboration 
is not their first choice, but they do not regard it as a 
major barrier either. In many ways, these statements 
are in line with the learning objectives of the projects, 
i.e. to give the students a balanced understanding of 
the potential and limitations of distributed 
collaborative work. 

CONCLUSION AND IMPLICATIONS 

The two pilot projects presented in this study illustrate 
how practical training in distributed collaboration can 
constitute an important part of the curriculum in 
graduate IS/ICT programmes. The students report a 
high learning outcome and perceived benefit from 
participating in this type of project. Task definitions 
and balancing of motivational factors were found to 
have a great impact on the groups’ efforts, sharing of 
workload and final outcome. Further, the projects also 
illustrate how technical noise and lack of experience 
with the technologies have represented barriers to 
effective collaboration, implying a need for training 
the students in integrated use of different 
communication services. 
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The study provides a valuable experience base for 
planning similar student projects in the future. In the 
following, the major implications for conducting this 
type of project are summarized In addition, we 
discuss some implications for practice that can be 
inferred from our results. Finally, although the focus 
of this study has been on building practical experience 
on training students in distributed collaboration, we 
also present some suggestions for further research 
related to distributed collaboration in general. 

Implications for Training in Distributed 
Collaboration 

Several implications can be drawn from the two pilot 
projects regarding design and conduct ofthis type of project: 

Group composition 

The experiences from the two pilot projects imply 
that the ideal group size would be two participants 
from each course. More than four members may 
result in "free-riding, and also represents a 
limitation when using tools like NetMeeting. In 
larger courses with a laige number of groups, the 
topics for the assignment can be duplicated if necessary. 

The distributed context needs to be ''reaV' 

The problems experienced by the two co-located 
groups in the second pilot project, illustrate the 
difficulty of "simulating" distributed collaborative 
work. Without real geographical dispersion, it is 
difficult for the students to restrict their 
communication to using electronic media only, 
when other group members are available for face- 
to-face contact next door, or even in the same PC lab. 

Selection of tasks 

In projects comprising members from different 
courses, it is vital that the nature of the task is 
cohesive, in the sense that it does not stimulate 
factions or polarization. Thus, tasks with an 
inherent "us-them" nature should be avoided. To 
reduce the students' uncertainty and enable them to 
quickly get started on the project collaboration, 
having a well-defined task seems appropriate. 
However, this needs to be balanced against the 
important learning process that may take place 
through "frustration" and problems related to 
project definition and delimitation of focus. 
Examples of tasks considered to be well suited are 
design of a simple software application, product 
evaluation (with no students perceiving any 



"ownership" of the products to be evaluated), or 
analysis of a collaboration problem and 
development ofsuggestions fortechnology support. 

Scheduling of the project 

The exposure of knowledge to "strangers, plenary 
presentation and gradin g of the project all 
constitute strong motivational fectors. Thus the 
students may easily end up using more time on the 
project than allocated. As illustrated by the first of 
the pilot projects, this may especially be the case if 
the task is equivocal in nature. The duration of the 
project period therefore needs to be adapted to the 
nature and complexity of the task, and the time 
available for the students to work on this. In 
general, initial hands-on training in the use of the 
different communication technologies will enable 
the student groups to arrive at a functional stage 
earlier. Further, both projects illustrate that finding 
time for synchronous meetings may be difficult. 
This can be eased somewhat by coordinating the 
course schedules for the different student groups. 
However, it is important that the students remain 
responsible for scheduling and coordinating the 
meeting activities. 

Establishing a common level of technical skills 
prior to the project 

A common platform needs to be established 
regarding basic skills in use of the technologies to 
be deployed. This may for example include 
practical training in the use of NetMeeting prior to 
the project. Although it is important that the 
students themselves discuss and develop routines 
for coordination of the work and use of the 
technologies, we recommend presenting some 
"practical advice" at the outset on how to 
effectively deploy the technologies in distributed 
project work. 

The students should be given the possibility for an 
initial face-to-face meeting 

The students reported that they perceived the 
initial face-to-face contact during the kick-off 
seminar to be of great importance for building 
relations, and believed this to have significant 
influence on the process and quality of outcome of 
the teamwork. If possible, we therefore 
recommend that the students be given this 
opportunity. This also seems to fit well with the 
practice that can be observed in "real" projects, 
where face-to-face meetings are given priority in 
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the early stages of a project. From a research 
perspective it could here also be possible to 
apply a design where half of the groups are 
given the opportunity to meet face-to-face for 
the first seminar, then comparing the process 
and outcome for these groups with groups not 
given this opportunity. However, this may 
introduce problems in grading the projects for 
the two categories. 

Balancing of motivational aspects 
To avoid any bias in motivation and efforts for 
students from different courses, the evaluation 
form and incentives should be made as equal as 
possible. This includes grading and credits given 
for the assignment in each course. 

Use remote supervision when needed 
One supervisor should be assigned per group, 
regardless of geographical location. Technologies 
for supporting distributed collaboration should be 
applied for remote supervision when needed. This 
may also add to the students’ experience from the 
project. 

Experience reports may stimulate the students* 
reflection on the work process 
By making experience reports part of the 
mandatory assignment, the students are stimulated 
to also reflect upon process aspects of the project 
work. However, it is important to have the groups 
assign a person responsible for keeping an activity 
log related to this, or else this may run the risk of 
getting low priority compared to the other project 
work. Further, the expected contents of these 
reports should be stated explicitly at the outset. 

Implications for Practice 

The setting created for these pilot projects is not far 
from work situations the students can expect to meet in 
their working life. However, we advocate caution in 
transferring results from an educational setting to a 
"real organization. We therefore only discuss some 
experiences that corroborate results from earlier field 
studies conducted in a real setting (Line, 1999a). 

Application sharing can be an efficient tool to support 
discussions related to various information objects like 
drawings and reports. The audio quality is still not 



sufficient for using NetMeeting as a standalone 
conferencing tool. This can be solved by using 
conferencing phones for the audio communication 
part. 

''Collaboration across distance demands competence 
and training to be efficient. This statement from one 
of the experie nee reports underpin s the need to 
establish procedures and structures for efficient use of 
the various technologies. Training in basic skills is 
critical in order to establish the right motivation for 
use. Without "hands-on” experience, very few are able 
to relate to and see the potential benefit of tech- 
nologies like application sharing and document 
repositories. Although there exist studies that can be 
seen as a first approach towards establishing best 
practice in this area (e.g., Mark et al., 1999), it is 
however difficult to define context free guidelines. 
The individual organization must develop its own 
procedures and use patterns based on its needs and 
accumulated experience (Line, 1999b; Munkvold, 
1999). 

Implications for Further Research 

There still exists little research on distributed 
collaboration that focuses on efficient, combined use 
of several collaborative technologies or services. We 
argue that there is a need for studies applying a holistic 
view on communication needs in distributed projects, 
looking into interdependencies and interplay among 
several services that together may constitute an 
infrastructure for distributed collaboration. This is 
accentuated by the rapid development of Mobile 
Internet services based on technologies like WAP, 
UMTS and Bluetooth, and is also exemplified by the 
new role taken by mobile phones as a supporting 
service in the second pilot project. A holistic view 
considering possibilities and limitations for integration 
and interoperability of different technologies and 
services is therefore necessary for understanding how 
new, efficientuse patterns can be developed. 
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ABSTRACT 

It is important for Information Systems graduates to have good interpersonal and communication skills as well as 
technical skills. Fostering these skills in the tertiary environment is not always easy. This paper looks at a way of 
using Joint Application Design (JAD) and co-operative learning techniques in the classroom to promotethe learning 
of the soft skills needed by IS graduates. An action research study was done using four case studies with the learning 
method being adapted between each. The study also investigated ways in which the method needed to be modified 
in order to deal with the diversity of students that were present in the classroom. 



INTRODUCTION 

Employers want IS graduates who can think, 
communicate, and work well with others. They 
also want individuals who have a good 
grounding in basic IS skills such as systems 
analysis and design and database concepts. 
These employers seem to be less interested in 
more specific technology skills (Van Slyke, 
Kittner & Cheny, 1998, p.lO). 

This was the conclusion reached by these authors after 
research done in America to determine the needs of 
employers for recent Information Systems graduates. 

The Information Systems ’97 Curriculum (Davis, 
Gorgone, Couger, Feinstein & Longenecker, 1997) 
recognises this need for the soft skills and has 
communication skills and interpersonal skills as two 



important characteristics of an IS graduate. 
Communications skills include listening, negotiating, 
facilitation, observation and presentation skills. The 
skills of leadership, small group communication skills, 
small group organisation and working with diverse 
people are listed among the interpersonal skills needed. 

Developing the soft skills of listening, communication, 
teamwork, conflict handling, debate and other 
interpersonal skills is difficult in a tertiary institution, 
especially as the learning of technical skills cannot be 
ignored. Group work is one way of fostering some of 
these skills but just putting students into groups and 
telling them to co-operate will not work effectively. 
Group work is particularly difficult in classrooms where 
there is a diversity of students. South Africa has a 
diverse population and a history of apartheid, which 
makes this diversity evenmore of a challenge. This will 
be described in more detail in the background section. 
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This paper reports on a series of case studies done over 
a number of years in South Africa which looks at how 
the techniques of Joint Application Design (JAD), used 
in industry, can be broug ht into the clas sroom in order to 
promote the development of communication and 
interpersonal skills while also learning the more 
technical modelling tools. For this study, four case 
studies were done. . The results of these case studies 
cannot be given in detail in this paper due to page 
constraints. An overview of what was found in each 
case study will be given together with a proposal for how 
the techniques can be used effectively. 

BACKGROUND 

This research took place in post-apartheid South Africa 
and some background is necessary to understand the 
context within which the research took place. There are 
11 official languages in South Africa. In the Eastern* 
Cape region, where this research was done, three 
languages are dominant, namely English, Xhosa and 
Afrikaans. The Xhosa speaking students generally 
come from disadvantaged backgrounds with inferior 
schooling. Apartheid has left South Africawith a legacy 
of poor schooling. Students, coming into the tertiary 
institutions, have often been taught by rote learning and 
learning from the textbook only. They find it difficult to 
work without a textbook orto engage in free enquiry and 
discussion (Ruth, 1996). While the government is 
trying to address this issue at secondary school level, 
currently these stude nts come into tertiary institutions at 
a decided disadvantage. It is thus left to the tertiary 
institutions to help students to bridge this gap. 



An additional difficulty is that the language used inmost 
tertiary institutions in South Africa is English, which 
may be a student’s second, or even third, language. 
Many of these students find it difficult to ask questions 
in class or to participate in discussions, especially when 
the whole class is present. Many are afraid that their 
language skills are inadequate or that their questions may 
be seen as naive by the other students in the class 
(Goduka, 1998). 

In South Africa, there are two types of tertiary 
institutions, namely Universities and Technikons. The 
Technikons, where this research was done, are career- 
orientedtechnical universities. The research was carried 
out at the Port Elizabeth Technikon and the Border 
Technikon (in East London.) The Port Elizabeth 
Technikon has a mixed student body with English, 
Afrikaans and Xhosa speakers. The Border Technikon 
was set up during the aparthe id years to educate African 
students and the student body is still almost exclusively 
Xhosa speaking. 

CASE STUDIES 

The research method used has been termed ‘Action 
Case’ by Braa and Vidgen (1999). The method consists 
of doing a series of case studies in order to get a deep 
understanding of the situation using both quantitative 
and qualitative methods. The results of these case 
studies are then studied critically in order to promote 
change. The method thus marries the interpretive and 
critical approaches to research. 



TABLE 1 

CASE STUDIES DONE 



Case Study 


Year 


Institution 


Class-Group 


Methods Used 


Languages Spoken 


1 


1998 


Port Elizabeth Technikon 


IS 2 


JAD only 


English, Xhosa, and 
Arikaans 


2 


1999 


Port Elizabeth Technikon 


IS 2 


JAD and coop 


English, Xhosa, and 
Arikaans 


3 


1999 


Border Technikon 


IS 3 


JAD and coop 


Almost exclusively 
Xhosa (92%) 


4 


2000 


Port Elizabeth Technikon 


IS 2 


JAD, coop and 
diversity 


English, Xhosa, and 
Arikaans 
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A series of four case studies was done with the method 
adapted after each case study. . This section gives an 
overview of the four case studies without trying to 
motivate the changes made. The next section will give 
an indication of how the learning environment was 
adapted and this will be motivated using the students’ 
comments and some of the quantitative results. 
Questionnaires, using both open and closed questions, 
were used to get a deep understanding of why the 
students answered as they did. An independentobserver 
was also used to tiy to get more understanding of the 
situation. 

Using Case Studies and having the researcher involved 
in the research limits the research in terms of 
generalisingthe results. For this reasonas much context 
as possible has been given to allow the readers to 
determine whether the method developed would be 
appropriate in their own context. 

Case Study One - 1998 

The first case study was used as a pilot study for the 
research. It was done during 1998 with the second year 
Information Systems students at the Port Elizabeth 
Technikon. One of the authors had been using JAD 
techniques in her classroom for a number of years but 
had not really studied whether they were effectiveor not. 

The techniques of JAD were taken from industry and 
incorporated into the classroom in order to help the 
students learn about JAD and modelling techniques. 

JAD, in industry, centres around a structured workshop 
where users from the different departments that will use 
a potential system are brought together with IT personnel 
to discuss their needs for the system. The workshop is 
controlled by a facilitator who keeps everyone ontrack, 
makes sure that everyone participates and that none 
dominate and ensures that the goals of the meeting are 
achieved. The facilitator is supported by a scribe, who 
documents the decisions taken. The meetings are 
structured around an agenda that is set up before the 
meeting. The participants must try to reach consensus 
about the needs of the system and any conflicts should 
be resolved in the meeting. These technique s and ideals 
were brought into the classroom where the students were 
given a scenario by the lecturer which they then had to 
model using JAD techniques and the modelling methods 
that they had been taught previously. 

One of the lecture halls with loose desks and chairs was 
fitted with five whiteboards situated around the 



classroom. The students chose their own groups or were 
put into groups as they came into the classroom. The 
lecturer gave the students a fairly complex scenario for 
which they had to model the functions and the data. 
One of the students acted as facilitator and another as 
scribe. These roles were rotated to give every student a 
chance. More details on the results from this case study 
can be found in Thomas (1999a). 

While the results of the case study were reasonably 
positive, there were some problems in the groups. 
Students complained about “cliques being formed, 
about non-participation by some and about difficulties 
with coping with the different languages. The students 
were English, Xhosa or Afrikaans speaking and had 
difficulties when other students would switch into a 
language that they could not understand. Some details 
of the students comments and other results are given in 
the next section. 

Someofthe problems thatthe students pointed out could 
be traced back to inadequate training on how to work 
effectively in groups. Other aspects could be directly 
attributed to the informal method of composing the 
groups which caused some people to feel left out of the 
group interaction. The subject matter was also not 
created to promote co-operation among the students. 

In order to address these problems, co-operative learning 
techniques were combined with the JAD techniques and 
tested to see whether they did improve the situation and 
reduce the problems experienced by the students. 

Case Study Two - 1999 

During 1999, a second case study was done where some 
of the techniques of co-operative learning were 
combined with the JAD techniques. This was once 
again done at the Port Elizabeth Technikon with second 
year Information Systems students. 

Co-operative learning should incorporate fiveprinciples, 
according to Johnson, Johnson & Smith (1991). These 
are positive interdependence, face-to-face interaction, 
individual accountability, social skills application and 
group processing monitoring. The circles of learning 
co-operative learning method was incorporated with the 
JAD workshop techniques, to enhance learning and 
promote co-operation. Knight and Bohlmeyer (1990) 
suggest eighteen steps that a circles of learning 
environment should adhere to. These are shown in Table 
2 . 
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TABLE 2 

INCORPORATING CIRCLES OF LEARNING AND THE JAD METHOD 

Recommended steps for circles of learning Implementation in JAD 

(Knight &^ohlme>^er, 199 0, p.2-3.) 



Clearly specify instructional objectives. 



Limit the group size to no more that six with smaller 
size groups for students that are new to co-operative 
learning to ensure that everyone will participate. 

Structure groups so that they are heterogeneous with 
respect to ability, sex and culture. 

Arrange groups in circles to facilitate communication. 

Use instructional materials that will promote 
interdependence among students. 



Assign roles to ensure interdependence. One could 
assign a summariser-checker to summarise the lesson 
and quiz group members; an encourager to encourage 
everyone to participate; a recorder to write down group 
decisions and an observer to make sure that the group 
collaborates. 



Explain the academic task.’ 

Structure positive goal interdependence by having the 
group produce a single product or by providing group 
rewards based on the individual performances of the 
different group members. 

Structure individual accountability for learning by 
giving individual tests or expecting the individual to 
explain the work. 

Structure inter-group co-operation. 

Explain the criteria for success. Explain how the 
individual grades work and how one can earn points for 
one’s group. 

Specify desired behaviours. These might include: 
Using people’s names, taking turns, making sure each 
person understands and agrees with the group’s answ er. 

Monitor students’ behaviour continually looking for 
problems with the task or with the collaborative effort. 
Provide task assistance. At times the teacher will need 
to intervene, clarify instmctions, encourage discussions 
or even teach. 
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This is done by giving the students the objectives of 
learning the modelling techniques as well as the 
objectives of learning to work with other people in 
small groups. 

Although the group sizes in the JAD sessions were 
sometimes eight or nine people, the use of the white 
boards and the roles that were played within the session 
allowed for easier communication and participation. 
Heterogeneous groups were used in the JAD sessions. 

The groups were arranged in a horsesho e with the white 
board at the front This facilitated good communication 
with the written work being done on the white board. 
The instructional materials (scenarios) were divided 
among the students so that different students were given 
the details of part of the system to be modelled. This 
was done to prom ote interdependence among the 
students. 

The students were given the roles of facilitator and 
scribe. Other students played the roles of users and IT 
personnel. The facilitator can be likened to the 
encourager and the scribe to the recorder. A 
summariser-checker was not needed as the lecturer 
played that role and an observer was not used for the 
second and third case studies, but was used in the fourth 
case study. 

The task was explained to the students. 

The group was required to model the Use Case models 
and the ERmodels on the board for the scenarios given. 
No rewards were given for this. 

The students were only tested individually at a later 
stage although the students’ group processing was 
evaluated by their fellow team members. 

This was not done. Each group worked on their own. 
Criteria for success were determined by the group effort 
as well as in the individual’sparticipation in the group. 
This was explained to the students. 

The desired behaviours were those that were expected 
for a JAD session in industry. These included making 
sure that people reached consensus, listening with 
respect to others and other desirable group behaviours. 
The lecturer did this. 

This was done to a certain extent although the lecturer 
triedto keep her intervention to aminimum. 
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Table 2 
(continued) 



Recommended steps for circles of learning 
(Knight & Bohlmeyer, 1990, p.2-3.) 

Intervene in order to teach collaborative skills of 
effective communication, building a trusting 
environment and managing controversy. 

Provide closure to the lesson with summaries by 
students and teacher. 

Evaluate the students’ work. Group or individualist 
incentives can be used although there should be some 
group incentive. 

Assess group functioning through ongoing observation 
and discussion of the group process. 



Implementation in JAD 



When problems in collaboration were seen, these were 
pointed out to the students. They were reminded of 
what they had previously learnt. 

This was not done with the class as a whole but the 
lecturer did it individually with each group. 

The lecturer evaluated the models of the groups and 
discussed any problems with the group. 

The groups assessed theirown group functioning using 
a questionnaire and the lecturerdiscussed any problems 
that they had with them. 



While the levels of participation, feelings of belonging 
and handling of language difficulties did improve, it 
seemed that there were still some difficulties being 
experienced by the Xhosa-speaking students. They 
were in a minority in the class and seemed to have 
difficulties in stating their opinions, arguing with the 
other class members and often felt that, although they 
tried to participate, the others were ignoring them. 
Details on this case study can be found in Thomas 
(1999b). Some of the results are also given in the 
following section. 

Xhosa speakers come from an inherently collective 
culture so one would not expect that they would have 
problems working in groups. Xhosa’s use the concept of 
"‘"‘ubuntu to describe humanness. This ethos is based on 
collectivism and is core to their culture. Goduka (1996) 
describes the following well-known Xhosa saying: 
""Umntu ngumntu ngabantu'* or in English, '"lam we, I 
am because we are: We are because I am. " This is 
indicative of the Xhosa culture where people work well 
in collective, pluralistic environments rather than in 
indivualistic environments. One would thus have 
expected that the Xhosa speaking students would have 
worked better in groups than their more Westernised 
counterparts. It was suspected that the difficulties were 
caused by the diversity of the students in the class and 
not by the Xhosa-speakers’ inherent culture . It was thus 
decided to do a third case study usingsimilar methods to 
the second case study atthe Border Technikon. Border 
Technikon has an almost exclusively black student 
population in their Information Technology course. 
Over 90% o f these stude nts speak X hosa with the others 



speaking one of the other eightindigenous languages of 
South Africa. 

Case Study Three - 1999 

The third case study was done during the second 

j 

semester of 1999 at the Border Technikon. Most of the 
groups were 100% Xhosa speaking and the students 
tended to speak a mixture of Xhosa and English. They 
would speak Xhosa and then use English words when 
referring to computer terms or when referring to terms 
used in the scenarios. Some of the groups had students 
who did not understand Xhosa andthese used Englishas 
the medium of discussion. The solution put onto the 
board was in English. 

The students really enjoyed these sessions and were very 
enthusiastic. Theirparticipation was good and they were 
overwhelming in their support for this type of activity. 
Many of them said that they were shy at first or thatthey 
were scared of feeling stupid at first but thatthey learnt 
a lot and felt confident later. 

It seemed therefore that the problem of the Xhosa 
speakers did not stem from their inherent culture, but 
rather from the diverse student population in Port 
Elizabeth. This was what had been expected. Ideas for 
dealing with diversity, and particularly diversity in 
groups, were then soug ht in order to determine how one 
could deal with the problem. As one of the objectives 
was to get students from different cultural groups to 
learn to work together, we did not want to use 
homogeneous groups in order to solve the problem. 
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Case Study 4 - 2000 

The last case study was done in the first semester of 
2000. This was also done at the Port Elizabeth 
Technikon with second year Information Systems 
students. Some of the preparatory work was modified 
to make the students aware of the difficulties of 
communicating and working with diverse cultural 
groups. The composition of the groups and the way in 
which the groups sat was also modified. Details of these 
modifications appear in the next section The scribe was 
also asked to act as observer and to let the group know 
if there was a problem with their group interaction. 

These changes are reflected in the method described in 
the following section, 

JAD TECHNIQUES IN A 
CLASSROOM WITH DIVERSE STUDENTS 

The students were divided into classes of approximately 
forty students each in order to facilitate the learning of 
theseskills and to promote group work and participation. 
It was felt that it was too difficult to work with groups of 
more than a hundred when doing this type of activity. 

The students were given different scenarios and asked to 
work together in a JAD session in order to create a Use- 
Case model and an exitity-relationship (ER) diagram for 
the system. 

Preparing Students for the JAD Sessions 

During the first case study the students were only given 
a little training in running meetings and in JAD 
facilitation. This did not seem to be sufficient to allow 
the students to handle group decision making and 
working co-operatively. Although most of the students 
found it a positive experience, there were quite a lot of 
complaints of people who dominated the sessions and 
others who did not participate. 

Co-operative learning literature suggests that putting 
students into groups and telling them to co-operate will 
not be effective (Johnson, Johnson & Smith, 1991), 
They suggest that the students should betaught some of 
the skills needed for co-operation before being asked to 
work in groups. 

A series of classes was set up to allow the students to 
learn skills like communication skills, listening skills, 
group dynamics and group decision-making skills. The 



students were also taught about how to run problem- 
solving meetings. Workshop-type activities were 
presented to help the students to develop these skills. 
Studentparticipation is imperativeduringthese sessions 
in order to give the students opportunities to practise the 
skills. As these skills are needed for IS developers, it 
was felt that they could be added to the curriculum. 
The JAD skills of facilitation and scribing were also 
taught to the students. These workshops were added 
from the second case study onwards. 

The second and third case studies had made us realise 
that the diversity of the students’ culture placed an 
important role in their group proces sing. As one of the 
English-speaking students in the second case study 
mentio ned; 

I think that JAD is a brilliant method of 
educating. If the members participated it would 
be beneficial. South Africa still has serious 
racial problems and language barriers . It is sad 
that students can ’t even do mock JAD without 
racial conflict. 

There is some debate in the literature as to whether 
students should be given explicit instruction about one 
another’s culture or not. Some authors (Bodibe, 1997; 
Koger, 1995) feel that students should be given such 
classes. Others like Goduka (1996) and Miller and 
Harrington (1990) argue that one should avoid a 
“tourist view of another’s culture as this makes students 
more aware of in- and out-groups. 

A middle road was chosen for this study. No specific 
instruction on different cultures was given, but while the 
students were doingtheir workshops on communications 
and group processing, special problems caused by 
diversity were highlighted. Problems that the lecturer, 
herself, had experienced when trying to converse in a 
second language were described, for example. Another 
example is the use of body language. The English and 
Afrikaans students were from a culture where one looks 
someone in the eye when talking to them. The Xhosa 
culture shows respect for a person by looking down. 
The idea of respecting one another’s viewpoint and 
listening with empathy to a speaker was also 
emphasized. 

During the fourth case study, the students were also 
given a workshop on being assertive. The observer in 
the previous case studies had noted that some of the 
students were passive while other tended to dominate. 
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Some also lacked the self-esteem to interact with the 
other students. The goal of the assertiveness training 
was to help the students to learn about having respectfor 
themselves and others and to teach them how to act 
assertively. 

Layout of the Classroom 

For the case studies in PortElizabeth,the classroom had 
been set up with five white boards around the classroom 
and each of the five groups were assigned a board. The 
desks and chairs in the classroom could be moved and 
the students sat in the classic horseshoe shape with the 
facilitator at the board. This is the method suggested for 
JAD session in industry (Carmel, Whitaker & George, 
1993). The horseshoe shape allows all the students to 
see one another and the board, thus promoting effective 
communication. The layout of the classroom is shown 
in Figure 1. The person standing at the board is a 
student, not the lecturer, who acts as the facilitator. 



In the third case study, the Border Technikon did not 
have any classrooms with this facility. Flipchart paper 
was stuck to the walls and the students usedthis instead 
of a white board. While the white board was more 
effective, as it allowed for easier modification of the 
models, the paper did not give the students too many 
difficulties and most groups did not have to do entire 
rewrites. 

During the fourth case study the observer noted, during 
the first session, that the students who sat next to their 
friends tended to hold private conversations with one 
another. One of the suggestions for JAD worksh ops is 
that the IT developers and users mingle around the 
horseshoe shape and do not sit nextto each other as this 
helps to reduce private conversations andpromote group 
cohesion. From the second session, the students were 
asked to move and mingle. This did seem to reduce the 
private conversations. 



FIGURE 1 

LAYOUT OF THE CLASSROOM 
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Group Composition 

During the first case study, the students had been 
allowed to form their own groups as they came into the 
classroom. This meant that students usually tried to 
form groups with their friends. As the later students 
came into the classroom, they would be put into the 
groups with the fewest number ofstudents. This method 
caused a number of problems and the students often felt 
that they were not accepted or that the others were 
forming “cliques . Language was also found to be a 
problem during the first case study. What would often 
happen would be that one particular language group 
would get into a group, for example, Afrikaans speakers. 
As the late comers arrived, the lecturer would add them 
to the group that had the fewest people and one would 
end up with a majority of Afrikaans speakers and one or 
two English or Xhosa speakers. As the majority of the 
group spoke Afrikaans, they would automatically switch 
to speaking their home language and the others would 
not understand them. This caused some friction within 
the group. As one of the students, commenting on what 
they disliked about working in groups, said: 

It way alw^ays difficult for me because we were 
forming our own groups, so if you don 't have 

those friends who know you, you struggle a lot. 

It was decided to borrow from the ideas of co-operative 
learning for the formation of the groups for the second 
case study. Jaques (1991) suggests that the ideal group 
size for co-operative learning is six as this allows for 
diverse ideas while still promoting the idea that everyone 
needs to participate. He suggests that the group should 
be structured in some way if it is larger than this. As the 
classroom had only five white boards, the students had 
to be divided into five groups which meant that the 
group sizes were normally about eight. This was not 
seen as a problem as the groups were structured and the 
use of the white boards does facilitate better 
communication within the group. 

There is some debate in the co-operative learning 
literature about the use of heterogeneous versus 
homogeneous groups with respect to academic ability, 
race and sex. As it was felt that getting the opportunity 
to work with people different from themselves was 
important, it was decided to use the heterogeneous 
approach. 



During the second case study, it was decided to create 
heterogeneous groups with respect to the students’ 
knowledge of different business areas that would be 
modelled, their academic ability and their home 
language. The business areas were chosen to get a wide 
spectrum of students within the groups; for example, a 
free clinic, a health club, and a library were used. 
Students from different language groups were grouped 
together but the students had to use English as their 
medium ofcommunication as that was the only language 
understood by all as well asbeing the language that they 
would have to use later when moving into industry. 

This method of dividing the students into groups did 
seem to be effective. None ofthe students in the second 
case study complained about feeling left out or about 
“cliques being formed. The quantitative results (shown 
in Table 3) also show a marked improvement in the 
students’ perceptions of their own contribution, the 
contribution of others and their acceptance within the 
group. The students also felt that they w ere more able to 
have their say in the groups. 

The students’ comments were mostly positive in the 
open-ended parts of the questions in Case Study 2 and 
they mentioned that people listened to them and that they 
were able to discuss their different points ofview. They 
seemed to enjoy working with otherpeople and hearing 
their ideas. They also enjoyed working asa team. There 
were still quite a lot of problems with people who did 
not contribute, however. There were also ten students 
who mentioned that they disliked it when others 
disregarded their suggestions. Upon further 
investigation, it was found that eight of these ten 
students were Xhosa speakers. This was disturbing. 
The observer also noted that the Xhosa-speaking 
students tended to be marginalized. 

These problems prompted us to do a third case study at 
Border Technikon where the Xhosa-speaking students 
showed no difficulties in working in groups, expressing 
themselves and making their opinionscount even in the 
groups thatcommunicated in English. As one can see by 
the quantitative results shown in Table 3, the 
participation of the students and their feelings of 
acceptance were also better than in the previous case 
studies. This led us to believe that the problems in Port 
Elizabeth were being caused by the diversity of the 
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TABLE 3 

PERCEPTIONS OF CONTRIBUTION AND ACCEPTANCE ACROSS THE CASE STUDIES 



When you were a Did you feel that Did you feel Did you feel 







group member and not 
the facilitator, did you 
feel that the facilitators 
gave you a chance to 
have your say? 


you were able to 
contribute to the 
group? 
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to the group? 


accepted as ‘ ^a 
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Vi 
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3,5% 
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Never 


0,9% 


0% 
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0% 



students rather than by the inherent cultural background 
of the Xhosas. 

For the fourth case study, some investigation was done 
into the formation of groups in a diverse society. Miller 
and Harrington (1 990) suggest that one should be careful 
of trying to get the ratios c orrect with respect to race and 
gender within a group as this may make the students 
more aware of their differences. Rosser (19 98) suggests 
that it may be intimidating for minorities who are alone 
in groups and that one should try to have more than one 



person of that minority in a group, even if it means that 
other groups have no minorities in them. The groups 
were made as heterogeneous as possible, but there were 
never fewer than two people in a group who spoke the 
same language, were from the same racial group or sex. 

Factors to promote group cohesion were incorporated 
into the fourth case study. For example, the students 
were asked to choose a name fortheir group. They were 
also told to make sure that they learnt one another’s 
names even when they were difficult to pronounce. 
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The students would write their names on the one side of 
the board and would try to referto one another by name. 
This is suggested by Bitzer and Venter (1996). 

These techniques seemed to be fairly effective. As one 
can see the students who felt that they were always 
accepted in their groups went up to 83,5% with the 
remainder (16,5%) feelingthat they were accepted most 
of the time. This was a distinct improvement from the 
first year where these figures had been 54,7% for 
always, 38,7% for most of the time and 6,6% for seldom. 

Many of the students commented in the open-ended 
questions about the positive experiences that they had 
working with diverse people. Some of the comments 
made w ere: 

'7/ helped me to learn how to interact with people of 
different races and cultu res. ", 

''It removed my shyness especially to different 
cultural groups and gender" and 
"It was good to work with the opposite sex, different 
races and coming into contact with different 
backgrounds . " 

The first comment was made by an English-speaking, 
white studentand the last two by Xhosa-speaking, black 
students. 

Instructional Materials 

During the first case study, the students were all given 
the same material. This allowed some students to sit 
back and not participate. In order to promote positive 
interdependence and individual accountability, co- 
operative learning techniques suggest that the study 
material be divided among the students in the group so 
that they do not all have the same material (Johnson, 
Johnson & Smith, 1991). 

An examp le helps to illus trate how th is aspect w as dealt 
with during the second, third and fourth case studies. 
The students were asked to draw a Use-Case Model and 
an ER diagram for the library, for example. Instead of 
giving all the students the same material, only an 
overview was given to all the students. The rest of the 
material was divided among certain of the group 
members who were termed the “users for the session. 
One of the mem bers was given the m aterial with details 
of what happens at the front desk, another for the 
ordering of books and a third for the financial aspects of 
the library. As the students were put into groups based 



on their knowledge of the different business areas, the 
students with the most knowledge of a particular 
business area would be made a “user in that particular 
area. This put each of the students into the situation 
where they had to participate, as they were the only 
people with the information needed by the group. It was 
also more comparable to what happ ens in industry where 
the JAD sessions are meant to bring together users with 
different knowledge. 

This seemed to have the desired effect, as shown in 
Table 3. The percentage of students who felt that they 
themselves always contributed in the group went up 
from 21.6% to 44.7% from the first to the second case 
study and then to 49,5% in the fourth case study. The 
percentage of those who felt that everyone else 
contributed went up from 6.7% in the first case study to 
17.4% in the second and then stayed fairly constant at 
17,3% in the fourth case study. The percentage of those 
who felt that the other people seldom participated went 
down from 26.7% in the first to 16.3% in the second and 
12,9% in the last. The participation of the more 
homogeneous group of students during the third case 
study, who were almost entirely Xhosa speakers, was 
65% for themselves and 55.2% for others in the always 
contributed categories. In other words, much higher than 
in the multicultural class. 

Running the JAD Sessions 

The students took turns at being the facilitator and the 
scribe. The student facilitator would stand at the board 
and record the design as given to him or her by the 
others in the group. The facilitator would also 
coordinate and control the session while the scribe 
recorded the solution and other decisions made on paper. 
The student facilitator was exp ectedto make sure that all 
the students participated, that none dominated and that 
the group stayed focused on their goal. If there was 
conflict, then the facilitator had to help the group to 
reach consensus and negotiate a solution. 

From the second case study, different students had 
different materials as they played the role of the various 
users of the proposed system. The facilitator had to 
draw this inform ation out of the students. 

The Role of the Lecturer 

The role of the lecturer was NOT to solvethe problems 
for the students, nor was it to get involved in any conflict 
situations. The lecturer’s role was more to make sure 
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that the JAD facilitatorwas doing his or her job. During 
the first case study, the lecturer concentrated on the 
modeling that the students were doing on the board, but 
from the second case study, she also monitored their 
group interaction as suggested by the co-operative 
learning literature. 

The lecturer monitored the groups to try to determine if 
they were working well together as a group. If the 
lecturernoticed, for example, that the group tended to all 
talk at once without listening to each other, then she 
would intervene and remind them to respect one 
another’s ideas and listen to one another. The lecturer 
would not act as referee, however. 

At the end of the session, the lecturer would go through 
the groups’ solutions with them and discuss any 
problems they might have had drawing the Use Case 
diagrams or the ER diagrams. 

Evaluation ofGroup Processing 

The co-operative learning literature suggests that 
students should evaluate their own group processing on 
a regular basis (Hamm & Adams, 1992), They suggest 
that students should be asked to determine where their 
group had problems and make suggestions as to what 
they needed to change in future. This was implemented 
using questionnaires from the second case study 
onwards. The students were asked to evaluate their 
group at the end of the first session; these questionnaires 
were consolidated by the lecturer who then gave 
feedback to the groups on their strengths and 
weaknesses. The groups were given these summaries at 
the start of the next class and discussed the suggestions 
made. 

The results were not summarized quantitatively. If, for 
example, one student in the group found the group 
unfriendly, this would be pointed out to the students, 
even if the rest felt that the group was friendly. It was 
felt that this was an indication that the students were 
leaving out one of their group members. The lecturer 
might then summarize as follows: 

While most of your group felt that the groupwas 
friendly, this was not the experience of all. 
Make sure that you include all your group 
members in your discussions. 

During the final case study the scribe was also asked to 
act as the observer and to let the students know if there 



were problems with the way in which the group was 
working. The students tended to find it difficult to be 
critical and would say that there were no problems even 
when the lecturercould see that there were. The use of 
the questionnaires seemed to be more effective in 
determining what problems existed in the groups. 
Something that one could consider would be to help the 
students learn about constructive criticism and how to 
offer constructive criticism as this might help them 
criticize more effectively. 

Assessment 

The Use Cases and ER diagrams were not given a mark 
by the lecturer. The solution was discussion with the 
students and suggestions forimproving it were made, but 
no marks were given to the group. The groups seemed 
to be happy with this. The students were tested 
individually on their knowledge of Use Cases and ER 
diagrams during subsequent tests and examinations. 
This helped to foster the individual accountability 
suggested by the co-operative learning literature. 

The students also gave each other a rating with respect 
to their participation in the group and their ability to act 
as facilitator. Various factors were given and the 
students evaluated one another on a scale of 1 to 5, This 
was done during the last session as it was felt that the 
previous sessions were for the students to gain 
experience. This peer evaluation was used as a small 
part of the grade for the students. 

LEARNING IN THE JAD SESSIONS 

The students from all three case studies felt that they had 
learnt. From the first case study, the students 
commented on how the method helped them to think and 
understand the work better and how it helped them to 
hear other people’s opinions. As one student said. 

It made one think and also consider another's 
point of view. The solution was seen through 
different angles. 

Another commented: 

The interaction, as well as arguing with other 
group members, really broadened your way of 
thought. 
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TABLE 4 

LEARNING GROUPS 
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Did you feel that 
the JAD tech- 
niques used in 
the classroom 
were effective in 
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learn how to feel 
more confident 
about how to act 
within a group? 



Do you feel 
that the JAD 
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Did not help at all 
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42,1% 


29,8% 


19,7% 


27,8% 


29,8% 


0,9% 


2,6% 


2,6% 


0,9% 



The students marks improved by approximately 9% in 
Case Study 2 and 8% in Case Study 4 from their test on 
the modeling techniques thattook place before the JAD 
sessions and their examination on the modeling 
techniques that took place afterwards. (While a t-test 
found the resultsto be highly significant, no methods of 
keeping the students from learning by any othermethods 
were used which is why no statistical results are given 
and it is left to the reader to decide if the results are 
significant.) 

Table 4 gives an indication of the quantitative results 
showing how the students experienced what they had 
learnt during the sessions from Case Study 2 to Case 
Study 4. As the questions were modified between the 
first and second case studies no comparison of results is 
possible. Note that one must be careful of saying that 
the method improved between Case Study 2 and 3 as this 
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was done with two totally different groups of students 
with different backgrounds. The students from Port 
Elizabeth Technikon had been previously exposed to 
quite a lot of group work in their othersubjects, whereas 
those from Border Technikon had not. 

It is interesting to note that the students’ perception of 
their learning of how to act in a group and how to 
interact with others, improved between Case Study 2and 
Case Study 4 both done at the Port Elizabeth 
Technikon. Those who felt that they had learnt a lot 
about how to act in a group increased from 48,8% to 
5 8,1%. Those who felt that it helped a lot to learn about 
interacting with other people went from 36% in Case 
Study 2 to 49,6% in Case Study 4. It seems that, in 
dealing with the diversity issue, which we had hoped 
would help with thestudents’ feelings of acceptance and 
contribution in the groups, we had also helped the 
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students to learn more effectively how to act in a group 
and how to interact with others. 

The students perceptions about their learning to work in 
groups, interacting with people and speaking in small 
groups are important as confidence has a lot to do with 
this ability. Many of the students commented on how it 
helped them to hear other people’s opinions or made 
them think better. 

On the soft skills side people spoke of how they felt less 
intimidated about sharing their ideas and how it helped 
them to realize that they should listen to other people. 
Many felt that it had helped them to overcome their 
shyness. Some comments made by some of these 
students include: 

It is difficult to start talking among strangers but 
this helped me see that in order to do the work, you 
must forget other things like what people think; 

When I worked in groups before l used to be shy or 
find that my ideas were not used because I would not 
say them out loud; 

It made me realize that anyonecan speak their mind, 
no matter who they are; 

It was a wake-up call for me to actually see what 
talking in front of people was all about. 

One of the students put it rather comically; he said: 
lam usually afraid of all those eyes looking 
at me but today I am not afraid of all those 
eyes, 

CONCLUSION 

The research has shown how JAD can be adapted to be 
effective in the classroom. The JAD workshops were 
modified using co-operative learning techniques and 
techniques for dealing with diversity to create groups 
where the students could learn to work with one another 
while also learningthe modelling techniques that are so 
important for Information Systems. 

While the research methods used limits the 
generalizability of the results, the context was set to 
allow the reader to make a decision as to whether the 
techniques used would be appropriate in their setting or 
not. Future research that might be useful would be to 



determine whether these methods are valid in other 
cultural contexts. 

Many of the skills mentioned in the IS Curriculum ’97 
(Davis et al., 1997) can be demonstrated in the JAD 
sessions. Each of the students has a turn to be the 
facilitator and must learn about controlling groups, 
handling conflict situations and leadership. All the 
students must communicate, negotiate and learn to work 
with people who are different from themselves. The 
students also learn about the modeling techniques as they 
argue and debate with one another, seeing their own 
misconceptions and helping one another. 

For quite some time, conventional systems 
development approaches have acknowledged 
the importance of the social element of ISD 
(Information Systems Development), 
Nevertheless, they concentrate on the technical 
process of systems development. They equip the 
developer with neither the tools, nor the 
knowledge, for dealing with thesocial processes 
intrinsic to ISD, Simple platitudes such as "get 
the support of senior management” or "involve 
the end user” are hardly sufficient to guide 
systems development. They tend to mask the 
social nature of ISD or portray it in simplistic 
ways. They do not allow developers to 
understand, let alone fully appreciate, thesocial 
nature of systems development (R 'lrschhQim & 
Newman, 1991, p30) (Brackets supplied.). 

This criticism can also be levelled at tertiary institutions. 
Most systems development life cycles taught in 
Information Systems curricula include the need to 
involve users and seek their support. The students are, 
however, never given the opportunity to practise the 
skills that they will need in order to do this. Using JAD 
and other active learning techniques can help the IS 
student to develop the interpersonal skills and 
communication skills that are so important to the 
Information Systems development process. 
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ABSTRACT 

With electronic commerce exponentially increasing, it has become important to understand how to effectively market 
to On-line and offline consumers. This paper examines which demand model is most appropriately used in electronic 
and non-electronic commerce for different combinations of business and individual transactions. A framework is 
developed to characterize which demand model, push or pull, is best suited for successful implementation of 
marketing strategy based on the type of transaction in electronic and non-electronic commerce. 



INTRODUCTION 

In the past few years, Internet activity has increased at 
an astounding rate (Berthon, Pitt, & Watson 1996; 
Wigand & Benjamin 1995; The Big Picture Geographies 
2000). Serving as a communication vehicle,the Internet 
now provides people with information ranging from the 
price of various computer parts, to books available for 
sale, to stock market activity. Because of the wide 
variety of available information, millions of people are 
now going on-line and surfing the Web. For companies, 
these people are all potential customers, and as a result, 
many companies and entrepreneurs have designed Web 
sites to sell their products and services to these 
consumers. With the emergence of the Internet as a 
dynamic medium for mediating transactions between 
individuals and businesses, the virtual marketplace has 
become a major focus for marketers. It has provided 
marketers with the opportunity to channel 
advertisements and information directly to consumers 



and to reduce advertising costs for the business 
(Hoffman & Novak 1996). 

In marketing, a primary focus is the exchange 
relationship (Hunt 1983). This relationship consists of 
three participants, buyers, sellers, and intermediaries 
(Gebauer & Scharf 1999). The roles of the parties 
involved and the methods of exchange they employ 
depend on a number of factors. This paper focuses on 
two of these factors. First, transactions in electronic and 
non-electronic commerce involve different 
combinations of businesses and individuals. This paper 
focuses on the following types of transactions: business 
to business, business to individual, and individual to 
individual. A second factor that is vital to a seller’s 
marketing strategy is the choice between the two 
demand driven models, the push and pull marketing 
models. Therefore, we will also examine which of the 
demand models is more ^propriate for electronic and 
non-electroniccommerce for the different combinations 
of business and individual 
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transactions. In the next section, we will conduct a brief 
literature review. 

BACKGROUND 

Bagozzi (1974 p.78) defined an exchange system as a 
’’set of social actors, their relationships to each other, 
and the endogenous and exogenous variables affecting 
the behavior of the social actors in those relationships.” 
The actors involved in the exchange process include 
salespeople, retailers, consumers, and marketers 
(Bagozzi 1975, 1974; Dwyer, Schurr, & Oh 1987; 
Morgan & Hunt 1994; Sheth & Parvatiyarl995). A 
common form of social exchange is business 
transactions. A business transaction is a process to 
exchange goods or services for some form of 
compensation (Gebauer & Scharl 1999). Transactions 
usually involve three categories of participants: buyers, 
sellers, and intermediaries. In a transaction, buyers and 
sellers play an active role because they exchange goods 
and services for compensation. Intermediaries offer a 
variety of services that facilitate and/or support the 
buyers and sellers (Gebauer & Scharl 1999). 

Following Gebauer and Scharl (1999), there are four 
stages in a transaction: information, negotiation, 
settlement, and after-sales and transaction analysis. In 
this paper, we are interested in the first stage in which 
both buyers and sellers are interested in searching for 
and gathering information (Gebauer and Scharl 1999). 
More specifically, buyers locate sources such as Web 
sites, magazines, and knowledgeable individuals to 
provide them with relevant and useful information 
necessary in making a purchase decision. Sellers also 
search for information. They are interested in where 
buyers go to collect information about products/ 
services, what qualities in a product/service are most 
important to the buyer, how buyers come to a final 
purchase decision, from where buyers prefer to 
purchase, i.e., electronic commerce or traditional 
commerce, and why. Once marketers have determined 
answers to such questions, they employ push and/or pull 
marketing models. 

The two primary demand models used in information 
exchange are the push and pull models (Ho 2000; 
Kalakota & Whinston 1997; Obendorf 1997). These 
models are opposing methods for sending information to 
consumers (Obemdorf 1997). In the push model, 
businesses strive to "get in the face” of the consumer. 
This might involve businesses sending unsolicited mail 
to the homes and businesses of consumers. Further, 



more recently information technology can enable 
marketers to go directly to the consumers’ desktops, e.g. , 
PointCast Inc. (Wagner 1997). Contrasting this, pull 
models are consumer-driven and controlled; thus, it is 
the consumer who decides that he/she wants to gather 
information (Katakota & Whinston 1997). So for 
example, an individual may decide to look at what other 
individuals are selling in newspaper advertisements. 

FRAMEWORK 

To facilitate the examination of these phenomena, a 
framework for analyzing the two forms of commerce 
and the three types of transactions was developed. The 
framework presents six cells in which the push and/or 
pull marketing model(s) are used the most. 



Forms of 

Commerce/Types of 
Transactions 


Electronic 

Commerce 


Non- 

Electronic 


Business to Business 






Business to 
Individual 






Individual to 
Individual 







The first dimension focuses on the two forms of 
commerce, electronic commace and non-electronic 
commerce. In the former, consumers are able to 
purchase goods and services over the Internet. 

As the Internet continues to expand at a rapid rate, more 
and more companies and individuals are choosing to 
conduct business via this medium. Thus, it is becoming 
increasingly important for those who wish to sell 
products or services over the Internet to be aware ofithe 
methods of exchange that complement consumers' 
purchasing behavior. This awareness will enable 
companies and individuals to sell their products and 
services more successfully over the Internet. In the 
traditional form of commerce, consumers purchase 
goods and service in physical space: in stores at the 
mail, at the convenience store down the read through 
newspaper advertisements, and at establishments located 
downtown. Because of the Internet’s increasing 
popularity, if these businesses hope to remain 
competitive using traditional channels, it is essential that 
they clearly understand how to sell to consumers in 
non-electronicconditions. Thus, even companies that do 
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not sell their products and services over the Internet also 
need to understand consumers’ behavior regarding the 
push and pull marketing models and which method of 
information gathering is more successful in 
non-electronic commerce. 

The second dimension of the framework focuses on the 
different types of transactions that occur in commerce. 
As previously indicated, these transactions include 
business to business, business to individual, and 
individual to individual. Presently, the number of 
business to business transactions over the Internet is 
skyrocketing. In 1 999, business to business transactions 
reached approximately $109 billion. This figure is 
expected to reach approximately $2.7 trillion by the year 
2004 (King 2000). Therefore, it is not surprising that 
business to business transactions are outperforming 
business to individual on-line transactions(Markets 
Advertising: Web Ads Take a B2B Approach 2000). A 
business to business transaction occurs when two 
businesses are involved in the exchange process. For 
example, Safeway may purchase goods from 
Pepsi-Cola. A business to individual transaction 
involves an individual purchasing products from a 
business. Most people are familiar with this type of 
transaction. For example, an individual purchases a 
bottle of Pepsi from Safeway. Finally, individual to 
individual transactions occur when one individual sells 
a good or service to another individual. For example, 
individuals often sell goods to others at garage sales or 
flea markets. 

ELECTRONIC COMMERCE 

The popularity of electronic commerce has grown at a 
phenomenal rate (The Big Picture Geograph ics 2000) . In 
fact, by the year 2002, it is estimated that there will be 
490 million people worldwide who are accessing the 
Internet. Further, the top fifteen nations in Internet use 
will account for 82% of all the users. This growth may 
be in part due to the benefits electronic commerce 
provides consumers, such as perceived convenience, 
information, and fewer hassles. As well, marketers have 
also benefitted because they can now make quick 
adjustments to market conditions, lower costs, increase 
audience size, and build relationships (Kotler 1999). 
Previous research (e.g. Ellsworth & Ellsworth 1997; 
Komenar 1997) has indicated that businesses that use 
the Internet as a medium for commerce rely on the pull 
marketing models for two primary reasons. First, some 
researchers (e.g., Berthon, Pitt, & Watson 1996; 
Ellsworth and Ellsworth 1997; Kotler 1999) believe that 



a business must bring its audience to its site, convince 
them to stay, and later have them return. Thus, the 
exchange process is customer initiated and controlled, 
i.e., the consumer, not the marketer, is in control of the 
interaction. A company has limited control over the 
number of potential consumers it attracts to its site. 
Second, new trends have emerged that negatively impact 
non-electronic commerce that uses push marketing 
models (Komenar 1997). More specifically, the increase 
in competition and consumers' expectations and the 
decrease in the effectiveness of the mass media suggest 
that companies must find new means to differentiate 
themselves from the competition while still fulfilling 
consumer's increasing needs and wants. Companies that 
use the Internet to provide a new medium to transfer 
information directly need to employ different strategies 
in selling goods and services. 

Although it is true that companies have to entice 
consumers to visit their Web sites, this is nothing new 
for companies involved in non-electronic commerce. 
For years, businesses and individuals in non-electronic 
commerce have been trying to find ways to encourage 
consumers to visit their establishments. Thus, almost 
any company that wants to sell a product needs to 
inform consumers that it exists. Non-electronic 
commerce has relied on media such as television, radio, 
magazines, newspapers, and billboards to promote 
brands or products. Recently, electronic commerce has 
begun to do the same. Companies on the Internet also 
push their marketing through other sources besides the 
Internet to increase consumer awareness to sell their 
products. For example, during the Super Bowl, several 
of the commercials aired and sponsors of the event were 
Internet companies. The visibility they achieved during 
this four hour broadcast may have been more effective 
than advertising on the Internet. Marketers are also 
finding other ways to advertise without having 
consumers first visit their Web site. Marketers now 
place advertisements on other Web sites, participate in 
forums and newsgroups, and send e-mails to potential 
consumers about the company's products/services and 
specials (Kotla* 1999). 

Regarding the negative trends that impact non-electronic 
commerce, companies on the Intonet can overcome 
some of these trends. As the amount of available 
information continues to increase at a rapid rate, the 
search for and identification of relevant and useful 
information is becoming time consuming and difficult 
for the consumer. Thus, there has been a growing 
consumer preference for channels that automatically 
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customize information to meet their wants and needs 
(Li, Kuo, & Russell 1999). On-line marketers are now 
having consumers sign-up to receive information via 
e-mail about products and services that are of particular 
interest to the consumer. For example, in business to 
individual transactions, if a consumer is interested in a 
certain technology, once the consumer provides the 
company with an e-mail address, the company can 
periodically e-mail the consumer information about this 
technology. In business to business transactions, 
businesses may also reach agreements so that the seller 
e-mails all relevant information to the buyer. Finally, 
marketers are taking advantage of new forms of 
technology that assist in informing consumers about 
products and services. One of the latest push 
technologies on the Internet is Webcasting. Webcasting 
enables on-line marketers to push and deliver 
advertisements and information to subscribers without 
the subscriber ever making a request (Kotler 1999; 
Mosley-Matchett 1997). Essentially, companies sip up 
with a Webcasting service, such as PointCast. These 
services will then automatically push customized 
information directly to the subscriber's desktop. Hence, 
in electronic commerce, the push model exists in two 
types of transactions, business to business and business 
to individual. Therefore, we hypothesize the following: 

HI: In both business to business and business to 

individual transactions in electronic commerce, 
the seller will prefer to use a push model to sell 
its goods or services to the buyer more often 
than a pull model. 

However, the pull model will be used in individual to 
individual interactions. Although millions of consumers 
go on-line everyday, the individual still has to attract 
and pull another individual to his/her site. To 
successfully attract consumers to a Web site, the seller 
would either need to create an attractive Web site or pay 
a search engine, like Yahoo! or Excite, to include his/her 
address in their lists of potential hits. However, for 
many individuals, this may require too much time and 
effort. Thus, many individuals have opted to sell their 
goods and services via intermediaries such as eBay. 
These intermediaries act as agents that stand between 
the buyers and sellers of a transaction and perfcrm 
necessary functions for the fulfilment of the transaction 
(Kalakota& Whinston 1997 ). These companies attract 
millions of potential buyers who can see the goods that 
people all over the world want to sell. Thus, to a certain 
extent, direct individual to individual transactions in 
electronic commerce are disappearing as people choose 



to employ the assistance of companies to sell their 
goods. Also, as consumers become more familiar with 
these Websites, they may stop surfing the Web to find 
other individuals from whom to buy. If the 
intermediaries provide reliable service and have a good 
reputation, consumers will use the services more 
because not only will consumers reduce their searching 
time, but they may also feel more confident purchasing 
a good from this type of company ratherthan a stranger. 
Like other businesses on the Internet, these companies 
use push strategies to increase buyers' awareness of the 
goods and services they provide. However, presently 
not all of the Internet intermediaries are pushing their 
marketing via other media Without this push, 
consumers still must surf the Internet to find the Web 
site. So, some of the intermediaries still rely on the pull 
model. Thus, one could expect that both types of models 
would be used in intermediary assisted individual to 
individual transactions. This leads to the following 
hypothesis: 

H2a: In electronic commerce, individual to individual 

transactions will rely on pull supply chains to 
sell goods or services independently. 

H2b: In individual to individual transactions that use 

intermediaries, either the push or the pull model 
may be used to sell goods or services. 

NON ELECTRONIC COMMERCE 

In business to business transactions,- the extent to which 
push and pull models are used will depend on the type 
of relationship that exists between the businesses 
involved in the transaction. With the development of 
technology such as EDI, strong relationships between 
companies have been formed. Because these companies 
have transactions with each otherexclusively for certain 
products, strong relationships will rely on the pull model 
for marketing. However, if a relationship does not exist 
between the two businesses involved in the transaction, 
the seller will rely on the push model. This will occur 
because the seller will be in competition with other 
businesses. Thus, this increased competition will 
emphasize the importance for the seller to push its 
marketing in order to increase the awareness of its 
product. More formally: 

H3: In non-electronic commerce, business to 

business transactions will depend more on the 
pull model when a strong relationship exists 
between the buyer and the seller; when this 
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relationship does not exist, conpanies will 
use the push model. 

Previous research (Kalokota and Whinston 1997; 
Komenar 1997) has indicated that in business to 
individual transactions, companies generally use the 
push model to sell their products. These techniques are 
successful in traditional marketing because it is easy for 
companies to "get in the face" of the a consumer. 
Companies have the option of numerous communication 
vehicles through which to market their product and 
brand. As mentioned above, they can use the television, 
radio, magazines, and billboards for marketing purposes. 
Therefore, we hypothesize that: 

H4: In business to individual transactions, 

companies using non-electronic commerce will 
use the push model to market goods and 
services. 

Finally, in individual to individual transactions, it may 
be difficult for a seller to find a potential buyer for a 
given product. This difficulty arises because individuals 
often do not have the means available to market their 
goods to the public independently. In this type of 
transaction, individuals need to rely on word-of-mouth, 
creative signs, and luck to sell their goods. Therefore, a 
pull model would be used. As in electronic commerce, 
individuals have the option of using businesses to help 
sell their goods. These businesses include suchmeans as 
newspaper advertisements, radio swap and shop call-in 
shows, and buy and sell magazines. However, unlike the 
Internet, these businesses only reach a limited number of 
peoples. As well, these media require the pull model. 
Therefore, the following hypotheses have been 
developed: 

H5a: Individual to individual transactions in 

non-electronic commerce will use the pull 
model to sell goods and services. 

H5b: Individual to individual transactions in 

non-electronic commerce that use inter- 

mediaries to assist in a sale will rely more on 
the pull model than individual to individual 
transactions that use intermediaries in electronic 
commerce. 

The framework for analyzing the forms of commerce 
and the types of transactions is completed as follows: 



Forms of 
Commerce/ 
Types of 
Transactions 


Electronic 

Commerce 


Non-Electronic 


Business to 
Business 


Push 


Pull/Push 


Business to 
Individual 


Push 


Push 


Individual to 
Individual 


Push/Pull 


Pull 



The extent to whidi the choice of marketing model is 
successful depends on consumers’ responses. This can 
be illustrated with the first initial attempt of business to 
individual transactions over the Internet. An overkill of 
unsolicited commercial e-mails resulted when 
companies employed the push model. The form of the 
push model was quickly viewed as an invasion of 
privacy and annoying for the consumer (Mosley- 
Matchett 1997). Companies have since responded to 
consumers. Although the companies still use e-mail as 
a major source of push marketing, the information that 
the companies send to. consumers is that which the 
consumer has indicated he/she would like to receive. 
This improved form of push marketing has resulted in 
only 1% of 1,000 recipients asking to be removed from 
a given mailing list (Obendorf 1997). 

CONCLUSION 

This paper has examined the need to take the form of 
commerce into account when examining the success of 
different demand models for business transactions. The 
Forms of Commerce/Types of Transactions Framework 
has been used to characterize the demand model, push or 
pull, best suited for successful implementation of 
marketing strategies in electronic and non-electronic 
commerce. For example, with all the new On-line 
businesses emerging there may exist companies that are 
unsure about which demand model would most 
effectively communicate its information to a consumer. 
Thus, management could examine the framework to 
determine that in a business to business transaction, 
unlike in non-electronic commerce where either push or 
pull models work, in electroniccommerce,apush model 
may be the best option. This same framework can be 
used to analyze other organizational strategies besides 
push-pull marketing demand models that examine 
transactions in electronic and non-electronic commerce. 
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Marketing is one of the central aspects of a business. It 
is used to inform consumers about and attract them to a 
company's products and services. Therefore, it is 
important for sellers to use the most effective demand 
model available in marketing strategy. The framework 
presented is especially beneficial to businesses and 
individuals who have or are deciding to sell products 
and/or services over the most repentant explosive 
medium, the Internet. For example, ifbusinesses realize 
that push marketing is the more effective demand model 
for electronic commerce, they can use push strategies to 
market their products/services. This framework is also 
helpful for businesses that rely on traditional forms of 
commerce. In non-electronic commerce, business to 
business transactions are not restricted to push 
marketing. Rather, both push and pull demand models 
can be used to inform the consumer. The option to use 
either model provides non-electronic commerce 
flexibility in marketing strategies. A deeper under- 
standing of the effectiveness of the demand marketing 
models in different types of transactions will facilitate 
almost any business or individual interested in selling a 
product and/or service. Whether the seller uses 
electronic or non-electronic forms of commerce, 
comprehending which demand model works best will 
assist in maximizing demand for products and services. 

Our research is not without its limitations. First, this 
framework may not directly apply to companies that 
hold monopolies in the market. It is clear that companies 
that do not have any competition may rely on either 
form of marketing demand model. Second, the 
framework makes the assumption that people are 
responding favorably to marketing via the new medium, 
i.e., the Internet. Statistics suggest that positive 
responses to electronic commerce do in fact exist. For 
example, in the United States, there are expectations that 
in the next 12 months, 79% of individuals who do not 
currently purchase products/services over the Internet 
will make their first on-line purchase (Stores January 
2000). Finally, we have neither tested the inherent 
assumptions of the theoretical framework nor its 
implications in an empirical fashion. However, given the 
importance and significance of understanding the effect 
of marketing strategy in both electronic and 
non-electronic commerce, our framework provides a 
positive first step. 

Future research should look at the other side of the 
information exchange process. Understanding 
consumers' reactions to the use of certain demand 
models in the different types of transaction in both 



electronic and non-electronic commerce would be 
beneficial. By considering both the buyers and sellers, 
the most effective and efficient demand methods could 
be used. 
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ABSTRACT 

This research examines the effect of hypertext and animation in the context of online learning. With an increasing 
number of online courses and degrees offered through the Internet and a rapidly increasing enrollment in such 
courses, it is important to assess and understand how these online features can affect or contribute toward learning. 
We propose and test a model to explain and predict the effect of hypertext and animation on online learning. 
Preliminary results of the study are reported. 



INTRODUCTION 

Currently, the Internet dominates in the development of 
information and communication technology. The 
Internet has taken an important role in our daily life by 
providing a wide range of services including 
entertainment, education and business. In the field of 
education, the Internet is used not only to supplement 
classroom teaching, but also as an increasingly popular 
medium for delivery of online education courses. An 
immense amount of research literature and Internet- 
based teaching modules are added to this network 
continually. These resources aim towards being a viable 
alternative for distance learning. According to a survey 
by the U.S. Department of Education, a total of more 
than 54,000 online education courses were offered in 



1998, with 1.6 million students enrolled (Lewis et al. 
1999). Such online courses continue to proliferate very 
quickly. 

Advances in Internet technology have provided users 
access to not only text and graphics but also digitized 
audio, video signals, and animations as well. Two online 
features that have made the Internet more adaptable for 
education and learning are Hypertext and Animation. 

“Hypertext” is the organization of information units into 
connected associations that the users can choose to relate 
at the click of a mouse. Hypertext has been found to be 
an effective method of training because it provides user 
flexibility and control over the method, speed, location, 
and order of information access (Marshall and Shipman 
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1995). In this way it caters to a wide range of users who 
have different goals, interests, requirements and 
comprehension abilities. Hypertext not only allows 
students the flexibility to access class information 
discretely in order to match their pace and personal 
requirements, but also furnishes students with a 
knowledge domain through which they can gain/retrieve 
information at their own pace to match their study order. 

Research has shown that individuals comprehend 
information better with visualization than with written 
text. One of the simplest forms of visualization is 
“Animation”. Ajiimation refers to computerized 
simulation of processes using images to form a synthetic 
motion picture. In the context of learning, Pezdek and 
his colleague (Pezdek and Stevens 1984; Pezdek 1987) 
predict that the use of the visual mode of communication 
effects the human mind to have a higher grasping and 
retaining capability. Animation is also expected to 
contribute toward learning since it appeals to the power 
of the human visual system (Clary, 1997). In Kehoe's 
(1996) review of studies on animation in education, 
visual aids are found to have a dramatic positive effect 
on learning if certain conditions (“explanative text”, 
“sensitive tests”, “explanative illustrations”, 
“inexperienced learners”) are met (Mayes 1989). 

The Internet and its related technologies have provided 
us with not only a new dimension in education, but also 
a wider range of teaching and learning styles. With an 
increasing enrollment in web-based courses and the large 
amount of resources invested in designing such courses, 
it is important to investigate how Hypertext and 
Animation can influence students' learning behavior and 
experience, and their subsequent impact on effectiveness 
of learning. Using the concept of Flow 
(Csikszentmihalyi and Csikszentmihalyi 1988; Hoffman 
and Novak 1994), we construct a model to explain the 
effect of hypertext and animation in the online learning 
environment. Empirical studies are being carried out to 
test this model. 

FLOW THEORY AND CONCEPTS 

Flow is the feeling or sensation of enjoyable experiences 
and the process of optimal experience (Csikszentmihalyi 
and Csikszentmihalyi 1988). Understanding flow is 
important in all disciplines and systems — if a discipline 
or system can provide the users with sensation of 
enjoyable experiences, then the users will voluntarily 
gain increasingly more information from the system and 
take a more active role in participation. In this manner 



the more the users enjoy and learn from the experiences 
the more will be the evolution of the discipline. There 
have been various definitions and descriptions given to 
flow. One such description is (Csikszentmihalyi and 
Csikszentmihalyi 1988 p. 29): 

Artists, athletes, composers, dancers, scientists 
and people from all walks of life, when they 
describe how it feels when they are doing 
something that is worth doing for its own sake, 
use terms that are interchangeable in their 
minutest details. This unanimity suggest that 
order in consciousness produces a very specific 
experimental state, so desirable that one wishes 
to replicate it as often as possible. This parti- 
cular ''state** is given the name "flow.** 

Privette and Bundrick (1987) have defined the same 
term as 

... an intrinsically enjoyable experience, is 
similar to both peak experience and peak 
performance, as it shares the enjoyment of 
valuing of peak experience and the behavior of 
peak performance. Flow per se does not imply 
optimal joy or performance but may include 
either or both. 

Challenges and skills are the universal preconditions of 
flow. For flow to occur, it is necessary to have a balance 
between the level of challenges and the level of skills 
possessed by an individual in a situation 
(Csikszentmihalyi and Csikszentmihalyi 1988), and for 
the flow to be sustained for long these challenges should 
"become more complex" (Csikszentmihalyi 1982) over 
time. Csikszentmihalyi (1982) uses the example of a 
^ tennis player to illustrate and support the concept of 
flow. According to Csikszentmihalyi, a tennis player 
who enjoys the game will improve his/her skills through 
playing tennis. Now if the challenge imposed by the 
opponent of this player does not increase/improve, the 
player will get bored and eventually lose interest unless 
he/she finds an opponent who offers challenges that meet 
the improved skills of this player. 

Researchers have suggested that flow is a useful 
construct for describing our interactions with computers 
(Csikszentmihalyi 1990; Ghani 1991; Ghani and 
Deshpande 1993; Webster, Trevino, and Ryan 1993). 
According to Hoffman and Novak (1994), control, 
content, and motivational characteristics influence four 
direct determinants of the Flow state: 1) Perceived 
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congruence of skills and challenges, 2) Focused 
attention, 3) Interactivity, and 4) Telepresence. Their 
model consists of an environment having various exit 
points. Only perceived congruence (of skills and 
challenges) and focused attention are examined in this 
research. The "perceived congruence of skills and 
challenges" condition is the prerequisite for flow to 
occur. Once the congruence between skills and 
challenges is achieved, flow is initiated. It is important 
to note that in order to sustain this flow state, congruence 
should always be present. During the flow state, the user 
experiences enjoyable feelings. At the same time this 
environment stretches a user's capabilities in learning 
new skills and enhances his/her “self-esteem and 
personal complexity” (Hoffman and Novak 1994). Any 
disparity between skills and challenges will result in the 
user either exiting the activity or selecting a more 
congruential activity (i.e., an activity where the user 
perceives a balance between his/her skills and the 
challenges offered by the system/activity). In other 
words, for a system or an activity to provide flow to its 
users, it should be flexible enough to continually match 
the skills of the involved users with challenges imposed 
by the system/activity. The system or activity has to 
cater to the improving skills of the involved users as they 
continue using the system or performing the activity. 
Focused attention is necessary to induce flow. Focused 
attention is defined as the "centering of attention on a 
limited stimulus field" (Csikszentmihalyi 1977). 
According to Csikszentmihaly/ and Csikszentmihalyi 
(1988), when one is in flow, “one simply does not have 
enough attention left to think about anything else”. The 
concentration is so immense that the individual does not 
have any thoughts of being happy or being sad or in fact 
anything else but the activity. The following is 
Csikszentmihalyi and Csikszentmihalyi's (1988) 
description of the flow experience of a young basketball 
player. While playing the game, s/he completely 
concentrates in it. Everything that matters to this player 
is the court. The player forgets or lays aside any other 
thoughts and feelings of problems such as fighting with 
his steady girl. This is due to involvement in the activity 
where the mind is totally concentrating in that activity. 
Flow can never be experienced if there is no complete 
involvement or concentration in an activity. 

Interactivity is the availability of immediate feedback 
between entities. This exchange of information and 
feedback is in form of a sensory dialogue. It is important 
for an activity to be interactive to induce and maintain 
flow. In terms of human computer interactions, 
interactivity can be thought of as an activity where the 
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user requests some action to be performed and the 
computer responds to that request by taking the 
appropriate action or displaying the results to the user. 
An example is that of performing a search on the web 
using a search engine. To carry out the search, the user 
will first enter the text to be searched. This text forms 
the input from the user to the computer. The search 
engine would respond to the user's request by presenting 
the search results to the user. The system and activity is 
interactive because the system output depends on the 
results of the search. Since our system is a "static" 
learning module, we will not examine interactivity in 
this research. 

Telepresence is the feeling of being present in a place 
different from your immediate physical location. Such 
a feeling is achieved while interacting with a medium 
such as one involving virtual reality. As hypertext and 
animation are expected to produce minimal effect, if any, 
on telepresence, we will not include it in this research. 



FIGURE 1 

RESEARCH MODEL 




PROPOSED MODEL 

Our research question is: “How does the inclusion of 
hypertext and animation effect flow in the online 
educational environment and what are their implications 
on effectiveness of learning (measured by performance 
in a test)?” Our model will focus on the impact of 
hypertext and animation on skills, challenges, focused 
attention, flow, and test performance. 
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Our model hypothesizes that: 

(A) Hypertext decreases challenges, and 
increases perceived skills, focused attention, 
flow, and effectiveness of learning 
(measured by test performance). 

We hypothesize that hypertext will lead to improved 
perceived congruence of challenges and skills by 
decreasing challenges in learning the domain and 
increasing users' perceived skills in the domain. With the 
availability of hypertexts that provide explanations on 
difficult terms, the challenge in learning the domain is 
decreased. Since hypertexts reduce the difficulty in 
learning the domain by providing explanations on 
difficult terms, users perceive themselves to have a 
higher level of skills in the domain. When learning a 
new or challenging domain, the perceived congruence 
between challenges and skills will be improved by 
decreasing the perceived challenges and increasing the 
users' perceived skills in the domain. By increasing the 
ease and convenience of accessing explanations on 
difficult terms, users are also more likely to stay focus 
on the learning process and in using the learning module. 
When perceived congruence (of challenges and skills) 
and focused attention are achieved, flow occurs. With 
greater flow and focused attention on the learning 
module, effectiveness of learning will improve. 

(B) Animation decreases challenges, and 
increases perceived skills, focused attention, 
flow, and effectiveness of learning 
(measured by test performance). 

Similarly, providing animations of the concepts covered 
in the learning module decreases the challenges in 
learning the domain and increases the users' perceived 
skills. Since the domain is made more interesting to learn 
through animations, focused attention will increase, 
which further leads to flow. We hypothesize that 
effectiveness of learning will improve by providing 
visualizations of difficult concepts in the domain and 
through increased flow and focused attention. 

RESEARCH DESIGN AND METHODOLOGY 

Figure 2 presents the proposed experimental design for 
the study. An Internet teaching module — developed for 
the department of Agronomy at the University of 
Nebraska-Lincoln — will be adapted for this experiment. 
The module presents the fundamentals of plant genetics 



and is targeted towards distance learning students in the 
field of Agriculture. For this experiment, we will recruit 
only novice subjects who have no prior knowledge of 
plant genetics. These subjects will be randomly assigned 
to one of the four experimental conditions. 

FIGURE 2 

EXPERIMENTAL DESIGN 



No 

Hypertext 



Hypertext 



No Animation Animation 



The teaching module presents basic principles of plant 
genetics and provides a glossary of terms in plant 
genetics. In the hypertext version of the module, the 
users only need to click on the term (or more 
specifically, the hypertext link associated with the term) 
to access explanations on the term. The explanations are 
presented in a small Internet browser pop-up window. 
All the information that is available through these 
hypertext links is also available in the system's “glossary 
document". In the non-hypertext version of the module, 
the users will have to separately access the "glossary 
document" and search for the specific term to access 
explanations on the term. Thus, the provision of 
hypertext links facilitates the search process by making 
the explanations more “easily accessible.” 

In the animated version of the teaching module, 
animations of various processes are presented along with 
the text. These animations are in addition to the regular 
graphics, diagrams and photographs that are embedded 
in the text. In the non-animated version, these processes 
are shown in a static manner. Thus, the use of 
animations provides a richer mode of communicating 
and expressing the textual and “static graphical” 
information to the users. 

Study data will be collected using a questionnaire 
administered at the end of the study and through 
computer logs. 
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PRELIMINARY RESULTS 

We analyzed the data collected using MANOVA. 
Interestingly, none of our hypotheses are supported. 
Contrary to our hypotheses, hypertext has a negative 
impact on the dependent variables in the study (p<.05) 
while animation does not produce any significant impact 
on the dependent variables. We suspect the reason may 
be due to the small statistical power of the study. Even 
with a large effect size (1^.40), the power attained is only 
.60. When analyzing at a medium effect size of f-.25, 
the power is only .28. Hence, the contrary and 
insignificant results may be due to Type II error. In 
future research, we will carry out this study with a larger 
sample size to assess the effect of hypertext and 
animation on the dependent variables proposed. 

EXPECTED CONTRIBUTION 

This research assesses the degree to which Hypertext and 
Animation contribute toward learning in the online 
environment. More specifically, it examines how 
Hypertext and Animation influence challenges, 
perceived skills, focused attention, flow and 
effectiveness of learning. From a research standpoint, 
this study investigates and explains how the provision of 
hypertext and animation impact on effectiveness of 
learning. From an application standpoint, the results will 
be extremely useful to educators, designers, and 
developers of web-based training courses. With an 
increased understanding of the effect of online features 
on learning, web designers and programmers can 
develop better and more effective online training 
modules by focusing their time and attention on 
incorporating features that contribute toward 
effectiveness of learning. 
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ABSTRACT 

The possibility of achieving significant economic growth through technological leapfrogging seems exceptionally 
attractive to developing countries. It appears that the possibility of developing countries adopting advanced 
technology immediately, rather than proceeding by mastering intermediate technologies in turn, is dependent on the 
type of product (specific technology) in question, the social condition and environment of the country in question. 
The ideas and thoughts raised in this paper are considered preliminary, and should be tested in practical projects 
in developing countries that have a technologically backward setting. 



INTRODUCTION 

The advances in information technology have been very 
rapid and significant over the past decades. The cost- 
performance of this technology has followed adeclining 
trend with each level oftechnological advancement, that 
it made the once “bulky and government-affordable” 
computer model a ubiquitous item in our workplace and 
lifestyle today. One of the notable technological 
breakthroughs in information technology is the 
convergence of computer technology and 
telecommunication technology. Thishas brought about 
a significant growth in network computing, enabling 
information to be processed and transmitted at 
unprecedented speed and time. Today, the types of 
network enabled by information technology vary in 
scope of coverage. The organisation could adopt 
information technology to create a framework that 
supports intra-organisational linkages for an integrated 
organisational business system. Such integrated 
organisational system characteristic could be extended 
beyond the organisation to interlinkages that involve 
affiliated offices, headquarters, suppliers and clients in 
different locations. This networking capability of 
information technology has brought about applications 
that go beyond the traditional mode of automating 
internal transaction process. It has already made 
significant impact on customer service improvement, 



costs reduction, productivity improvement, new industry 
structure and relationship, new information technology 
based products or services, as well as enabling a new 
dimension in the marketing of product (Fong 1999). On 
the macro level, information technology has the potential 
in cr.eating national cohesive relationship and extending 
this relationship to the global level, and opening a new 
horizon of opportunities. This would speed up 
globalisation and intensify competition among nations. 
This potential of information technology has triggered 
off the pursuit ofinformation technology adoption in the 
developed nations, in industrialising countries and in 
some of the trade-oriented developing countries. 

Currently, these different groups of countries are at 
different stages oftechnology adoption. The developed 
nations are comparatively more established and 
experienced in the adoption and application of 
information technology because they were the early 
adopters ofinformation technology. Their experience 
in the adoption and application of information 
technology generally involved an evolutionary process, 
in which these countries tend to rep lace their superseded 
technology with the new and advanced technology as the 
latter emerged in the market place. The developing 
countries were, on the other hand, late adopters of 
information technology and are more technologically 
backward than its developed counterparts. With the 
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relativeease of access to advanced technology in today’s 
world', this raised the question on the possibility of 
developing countries (whichare stillusing traditional or 
backward technology) adopting advanced technology 
immediately (termed as “technological leapfrogging”) 
rather than proceeding by mastering intermediate 
technologies in turn. Successful technological 
leapfrogging has significant impact on the 
technologically backward countries as they are exposed 
to unprecedented opportunities offer by the new and 
advanced technology. This paper is established in the 
interest of determining the possibility of technological 
leapfrogging by developing countries. It lays out those 
issues surrounding the concept and the possibility of 
technological leapfrogging, and is aimed to serve as a 
initial ground for further research effort in this area. 
This possibility of technologicalleapfrogginghas major 
implications for information systems pedagogy and 
curricula in the developing countries since they are 
important for equipping users with the skills and 
knowledge to handle the respective technology adopted 
for usage. 



TECHNOLOGICAL LEAPFROGGING BY 
TECHNOLOGICALLY BACKWARD COUNTRY 

The concepts of evolutionary process of information 
technology adoption and technological leapfrogg ing are 
illustrated in Figures 1 and 2. Figure 1 provides a 
simplified illustration of the evolutionary process of 
information technology adoption in the developed 
countries. This illustration was constmcted on the 
assumption that the stages of technology development 
could be presented by a neat series of steps and that 
higher technological superiority is attached to each 
higher stage of technology development. The diagram 
shows that costs are incurred by the technologically 
advanced develope d countries in their active adoption of 
each newly emerged intermediate level of technology 
and in the displacement of their superse ded intermediate 
technology structures that were incompatible to the 
newly emerged technology. Figure 2 portrays the 
concept of technological leapfrogging, in which a 
technological backward country skips intermediate 
technologies and goes straight to the latest state-of-art 
technology. 



FIGURE 1 

ADOPTION OF INFORMATION TECHNOLOGY IN DEVELOPED COUNTRIES 
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FIGURE 2 

TECHNOLOGICAL LEAPFROGGING BY DEVELOPING COUNTRIES 



Stages of 
Technology 
Develop inent 




Because the technologically backward country hasbeen 
inactive in the adoption of each newly emerged 
intermediate level of technology, it has the option of 
adopting either intermediate technology or latest‘state- 
of-the-art’ technology (technological leapfrogging) in 
its late decision to invest in technology for production. 
According to the World Bank and the Asian 
Productivity Organisation (Hanna et al. 1995 and APO 
1990), information technology provides developing 
countries with the opportunity to accelerate economic 
development. The advent of this technology to each 
higher stage level is commonly associated with greater 
technological capability potential. Adopting the latest 
‘state-of-the-art’ technology would mean that this once 
technologically backward country is exposed to the 
unprecedented opportunities offer by the new 
technology. These unprecedented technological 
opportunities were not readily accessible to the 
advanced countries without incurring (intermediate 
technology) displacement costs in these countries, due 
to their previous intermediate technology structure. 
Thus, the late adopter in technology may be able to 
avoid incurring the time delays and the costs that were 
incurred by the advanced countries in their active 
adoption of each newly emerged intermediate level of 
technology. The possibility of achieving significant 



economic growth through leapfrogging to advanced 
information technology thus seems exceptionally 
attractive to developing countries. 

Research studies (Mody and Sherman 1990, Antonelli 
1991, Mody and Dahlman 1992, Lamberton 1994) 
noted that technological leapfrogging in the 
telecommunication and computing infrastructure is 
technically feasible in terms of physical facilities in the 
developing countries. However, what has remained 
elusive in this concept is the capability of the 
developing countries to harness fully the technological 
potential offered by the radical new technology for 
economic advantage. Research works in this area are 
scarce, hindered by limited availability ofempirical data 
from the developing countries and the relatively recent 
emergence of advanced information technology. It has 
been argued that empirical case studieson the capability 
of developing countries in technological leapfrogging 
are badly needed to answer the crucial question about 
whether the advantages of technological leapfrogging 
are within the reach ofthese countries (Ausubel 1991). 
According to Sharif (1989), massive resources and a 
long time span will be involved in gathering reliable 
data from the developing countries in search of the 
answer to this question. 
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The technological leapfrogging concept was first 
established by Soete (1985), from his observationofthe 
microelectronic industry which allowed rapid 
industrialisation within the adopting countries. The 
concept has received recognition of its bearing on 
information technology applications in the developing 
countries since the early 1990s (Antonelli 1991, Mody 
and Dahlman 1992, Lamberton 1994). Howe ver, there 
is limited evidence to determine the actual occurrence of 
technological leapfrogging inthe developing countries 
involving this new technology . This paper serves as an 
initial effort to contribute to this relatively new field of 
study of technological leapfrogging in information 
technology. 

RELEVANCE OF 

TECHNOLOGICAL LEAPFROGGING 

There are particular features of the information 
technology developments taking place in the past 
decade, which seem to increase the relevance of this 
concept to the developing countries. When the 
following features of the advanced information 
technology are contrasted against the intermediate 
information technology, technological leapfrogging 
seems to qualify as an inevitable move in infomiation 
technology adoption. 

Open Systems 

Operating systems that are established under open 
systems standards can be easily integrated for network 
formation. In otherwords, thetechnical choices of such 
systems for network establishment are not constrained 
by vendor, type or vintage, and this provides a high 
level of communication flexibility. Early information 
technology had been dominated by proprietary system 
standards, and the incompatibility among the different 
manufacturers’ equipment and software resulted in 
incomplete and inconsistent exchange of information. 
Adopting intermediate technology would mean re-living 
the problem of technical inflexibility thatthe advanced 
countriesare experiencing in the 1990 s, and which these 
countries are trying to overcome through open system 
in advancedtechnology adoption. The inability ofthese 
advanced countries to advance rapidly to new 
technology adoption is reflective of the inflexible 
technology infrastructure inherent in the intermediate 
technology design, due to the earlier pursuit of the 
proprietary concept in technology development. The 
new technology, on the other hand, had provided a 
greater degree of infrastructure flexibility^, in which 



consideration is given to the incorporation of future 
emerging technologies and migration to future 
technologies without incurring exorbitant costs in 
technology replacement. Inthe longrun, this flexibility 
means a lower cost in in vestment. The adoption of new 
technology therefore seems to be the most economic 
alternative, from a wider and longerterm in perspective. 

Demand on Users 

The potential of advanced technology may not be fully 
utilisedor exploited by the developing countries, but the 
learning paradigm offered by the advanced technology 
framework has more benefits than that of intermediate 
technology. Closely related to the advancement in 
technology is its reduced demand on the prior 
knowledge and experience of users in technology usage, 
which is known as the increasingly ‘user-friendly’ 
feature of systems. One obvious attribute of this 
phenomenon is the human-computer interface factorin 
technology developm ent, which is a rapidly developing 
area of focus in technology innovating countries. This 
emphasis is highly evident in funding criteria for 
technology R&D, training, design development and 
market demand (Australian Science and Technology 
Council 1990). Research studies show that an effective 
human-computer interface has a strong influence on 
learning curve, performance speed, rate of error and 
user satisfaction (Shneiderman 1992). It tends to share 
a strong positive correlation coefficient with the 
mentioned user-related variables. 

Globalisation 

The development of information technology 
applications and the revolutionary potential of 
information technology also do not suggest that 
interme diate technology is the ideal choice for adoption. 
The successful applications of advanced information 
technology in the developed countries and the Newly 
Industrialising Countries, such as the Internet, trade 
networks and banking networks, strongly project a 
future world economy where globalisation is 
underpinned by electronic global networks and an 
electronic global marketplace. This point isalso related 
to the inherent open system characteristic of advanced 
information technology. The adoption of intermediate 
information technology by developing countries would 
involve the heavy price of being excluded from 
integrating into the global market, especially when the 
concept of the GII (Global Information Infrastructure) 
is fully realised. Thus, these developing countries 
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would be in the position ofa perpetually technologically 
backward country, while the developed countries are 
advancing their vision of a global integration for 
economic development, through the displacement of 
intermediate technology for advanced technology. 

Thus in comparing the technical attributes of the 
advanced technology to intermediate technology, there 
seems to be stronger reasons to leapfrog to advanced 
technology adoption than to adopt intermediate 
technology in the developing countries. 

OPPOSITION TO 

TECHNOLOGICAL LEAPFROGGING 

The relevance of the technological leapfroggingconcept 
still remains unresolved for develo ping co untries . There 
are opponents who doubt the ability of the developing 
countries to take advantage of information technology 
leapfrogging (as found by Hanna et al. 1995). Those 
who oppose the idea of technological leapfrogging in 
information technology could make use of two 
perspectives drawn from technological leapfrogging in 
generic technology. One is that leapfrogging runs in 
contradiction to the accumulative nature of the learning 
process (Pavitt 1984, Hobday 1994); the other is that 
new technology has a strong tendency to undermine the 
social structure, and to dismantle the existing stock of 
human and physical resources (ILO 1985a & b, Wang 
1991). In other words, an ‘unprepared’ (lack of prior 
knowledge and experience) economy or society would 
not be able to cope with the new technologicalparadigm 
nor to exploit the potential it offers (Jian 19 95). 

However, it is noted that advanced technology does not 
necessary brings only hardship to the adopting entities. 
The social context of a developing country may turn out 
to favour technological leapfrogging. For example, the 
status symbol associated with owning a mobile phone 
has not only led to a quick diffusion of the mobile 
phone technology in the People’s Republic of China, 
but the social consciousness of being seen with the 
latest mobile phone model has driven technology 
leapfrogging in this area in the country. On similar 
ground, the pattern of mobile phone adoption fits the 
Chinese business and organisation environment well. 
This is because most managers of local Chinese 
companies do not often have the privilege to work in 
clean and nice offices. The mobile phone technology 
provides a mobile office environment for them to 
conduct businesses in more appropriate settings with 
their clients. Thus, advanced technology may not 



present the same challenges to organisations from 
different social environments, its unprecedented 
potential may befit the social condition of the 
technology adopting entities. 

CONCLUSION 

The possibility of achieving significant economic 
growth through technological leapfrogging seems 
exceptionally attractive to developing countries. It 
appears that the possibility of developing countries 
adopting advanced technology immediately, rather than 
proceeding by mastering intermediate technologies in 
turn, is dependent on the type of product (specific 
technology) in question, the social condition and 
environment of the country in question. The ideas and 
thoughts raised in this paperare considered preliminary, 
and should be tested in practical projects in developing 
countries which has a technologically backward setting. 
The author looks forward to similar efforts inexploring 
the possibility of technological leapfrogging in 
developing countries. 

ENDNOTES 

1. This is in contrast to the Cold War period during 
which some of the developing countries were 
specifically prevented from procuring leading-edge 
technology, such as computer hardware, computer 
software, telecommunications switching equipment and 
related technology for the possibility of a military 
buildup. 

2. The result of movement towards open systems and 
standards for system interoperability. 

\ 
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